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PREFACE 

The original impetus for this monograph was a grant from the New 
Jersey Basic Skills .ouncil of the New Jersey Department of Higher Education 
to the Graduate School of Education of Rutgers University. The purpose of 
the grant was to evaluate two college basic skills remedial programs, one in 
a four-year state college, one in a two-year community college, and to 
produce a handbook for conducting such evaluations. This handbook was 
produced, and distributed to all public colleges in New Jersey. Drawing 
upon the response and suggest ns from practitioners, the authors have 
produced this monograph for a more general audience. 

This * s nQt a text on con ducting program evaluation; it is a list 
of evaluative activities that you might be interested in undertaking. If 
combined with a text on evaluation which covered basic philosophical, 
theoretical, and design issues, this monograph would provide the "specifics" 
for translating generalities into the tasks wh'ch face the college basic 
skills program administrator who must actually conduct an evaluation. 

The examples used i.i the manuscript come from our experiences in 
New Jersey and therefore do not provide a broad spectrum of studies geo- 
graphically. At the end of this monograph is an annotated listed of 32 
studies conducted in colleges around the country. The authors hope that 
this report will be a useful supplement to your evaluation library and that 
its suggestions will help you in your evaluation activities. 

Jeffrey K. Smith 
Carl J. Schavio 
Donald B. Edge 
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OVERVIEW 

The process of evaluation involves assessing the merit or value of some 
person, idea, or process. Because merit and value are subjective notions, 
evaluation itself is inherently subjective. Since some of the evidence 
which is brought forth in most evaluations is of an objective nature, the 
entire process can appear to be more objective than it really is. Outcomes 
which are quantifiable are not necessarily the outcomes which are most 
cr it ical . 

Thus, evaluations are never the straightforward, black or white 
activity that one might wish them to be. There are, however, a variety of 
ways to gather evidence regarding the merit or value of an educational 
program which are readily communicable and widely accepted. The purpose of 
this paper is to present detailed descriptions of some of the ways that 
evidence can be gathered for the purposes of evaluating remedial basic 
skills programs in colleges. 

This paper is based on several assumptions which ought to be made 
explicit at the outset. First, the audience for this paper is the individual 
at the college who is responsible for evaluating the basic skills program. 
Often, this individual is the director of that program or reports to that 
director. For purposes of writing annual reports to boards of trustees or 
to funding agencies, there is an urge to frame the results as positively as 
possible. For purposes of improvement of the internal workings of the basic 
skills program, however, a 'warts and all" picture is necessary. It would be 
foolish to argue that "the data are tho data" and cannot be cast in flattering 
or detracting lights. On the other hand, certain types of evidence are 
more compelling than others. Well aware of what needs the evaluator has, 
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we will present evaluative techniques that are not prescriptive in nature, 
but rather a list from which techniques may be selected. 

Second, it is assumed that the reader has a rudimentary background in 
statistics (means, standard deviations, correlation), but not a sophisticated 
background . The more statistically sophisticated reader may wish to skip 
sect ions . 

Third, it is assumed that each reader has at least one evaluation 
technique or approach that is superior to many which are presented here. 
At least a half dozen scholarly journals are devoted to the study of 
evaluation: Evaluation Quarterly , Educational Evaluation and Policy Analysis , 
Evaluation, Studies in Educational Evaluatio n, Measurement and Evaluation in 
Guidance and Evaluation and Program Planning . Actually, this third assumption 
is really a caveat in assumption's clothing. This paper is not the be all 
and end all of evaluating college basic skills remedial programs. What it 
attempts to provide is a set of reasonable and fairly efficient techniques 
that can be employed in basic sk'lls evaluative efforts. 
The report consists of five sections: 

1. Program Documentation — a discussion of how to assess what your basic 
skills program is so that you might know what is being effective and 
what isn't. 

2. Placement Effectiveness — a discussion of how to determine whether 
your placement procedures are satisfactory. 

3. Program Effectiveness — a discussion of how to assess the effectiveness 
of your remedial program. 

4. Special Studies ~ a discussion of how to address special questions in 
>our program. 
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5. A Bibliography ~ sour^s for finding more extensive discussions of 
particular issues and examples of other colleges' efforts; 
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PROGRAM DOCUMENTATION 

In order to assess whether something is working well, it is beneficial 
to know what that something is. This is a fairly simple task when that 
something is say, a particular make and year of an automobile. The features 
and characteristics of the car are well-known, easily documentable, and 
fairly consistent from one car to the next. If one considers a play however, 
the problem is a little more complex. Casts vary; stages vary; props, 
scenery, and costumes vary. In assessing the merit of a play, there are 
variables which may affect the evaluation. 

Now compare a pLay to an educational program The play has characters, 
dialogue, m.J stage directions which are mostly invariant. Compared to an 
educational program, a play is a model of stability. Educational programs 
vary from one semester to the next, one instructor to the next, one student 
to the next. In order to understand the evidence that is provided in an 
evaluation, it is first necessary to understand the program that produced 
the evidence. 
History of the Program 

Some basic skills remedial programs have been in place much as they are 
today for ten or fifteen years. Others have gone through dramatic revisions 
and still others represent fairly recent endeavors. If a brief history of the 
program is presented in an introductory note to an evaluation report, it may 
provide a perspective for the efforts and achievements of the current year. 

A history of the basic skills program may include the following 
points: 

1. Impetus for the creation of the program. Was there a concern for 

admitting students less well-prepared than the student population at 

° i r 
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that time? Were the students who comprised the student body at the 
inception of the program deemed to be insufficiently prepared for 
college level work? Was a funding source found for doing remedial 
work? Did the impetus come from students, faculty, administration, 
the state? 

I. Original goals for the program. Was the program intended to be 

voluntary? Did the program consist of regular remedial classes or 
tutorial help? Were regular faculty used or were specialists in 
remediation hired? 

3. Development of the program. How has the program grown and changed 
since its inception? What were the influences on the change? How has 
the magnitude of the program changed in terms of number of students, 
number of faculty, number of contact hours, number of administrators? 

4. Status of the program within the college. What has been the status 

of the basic skills program in terms of how it is viewed by the various 
constituencies? What seem to be the cause3 of that status? 
If you have been associated with the program since its inception, 
writing a history may not be too difficult. There is, of course, a problem 
of selective perception. If you have more recently joined the program or 
wish to approach writing a history of the program as if you had, some 
sources of information would be: 

1. Previous evaluations. These are probably the best single sources. 

2. Funding proposals. These are particularly valuable or looking at 
original goals. 

3. Memos and other internal documents. Reports to superiors, suggestions 
for improvements, and so on, are good for providing evidence concerning 
goals and perceived status of the program. 
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4. Other college documents. Documents which refer to the basic skills 
program which were not written or j^o program personnel but which 
refer to the program may be helpful. Reports on "the state of the 
college, 11 accountability reports, and so on, may contain references to 
the program which are insightful. 

5. Old records. These help provide estimates of numbers where no of-ficial 
figures exist. 

6. Interviewing personnel". Both those associated with and not associated 
with the program can be helpful in understanding various aspects of the 
program. It might be worthwhile to find individuals who represent a 
cross section of experience with the program, and longevity at the 
college (student, faculty, administration). 

Once the evidence concerning the history of the program has be£n 
gathered, it must be weighed and condensed. This is a difficult task; the 
goal should be to produce a brief but illuminating picture of the program. 
When making decisions on what to include and not include, the key question 
to ask should be "Does this help the reader understand where we are now?" 
The Program Today . * 

The documentation of the program as it currently exists provides the 
framework for understanding the evaluation as a whole. Among the pieces of 
information that might be included in a description of the program are: 
I. What is the program? ^ 
A. What help is offered? 

1 . Through classes (a breakdown of courses and sequences ) 

2. Through tutorials 
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3. Other (Some colleges have programs where students are 

identified as having special needs and then are helped through 
their regular college courses,) 
B. Who is involved in instruction? 

1. Regular faculty 

2. Full-time basic skills faculty 

3. Co-ad jut int ins t rue tors — * 

4. Tutors 

5. Other students 
II. How large is the program? 

A, Enrollment - 

1. To al 

2. By sequence 

3. By course 

4. By section 

5. Tutorial or laboratory attendance 

B. Faculty (By breakdown in I.B. above.) 

III. Who is in the program? (This information can be presented for the 
group as a whole or could be broken down by course, subject area, 
subject level, and so on. 

A. Statistical breakdown of student demographics 

1. Sex 

2. Ethnicity 

3. Age 

4. Year of high school p^rollment 

5. Type of high school program 
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6, High school class rank 

7. Number of years of high school study in various subject areas 

B. Statistical breakdown oT student pre-instructional performance, 
at t itudes 9 expect at ion* and goals. 

1 . Means , standard deviations , of student pretest scores by 
course , subject area, or level 

2. Results of responses to attitudinai and self-report questionnaires 
such as: 

Perceived need for course by student 
Courses taken in high school 
Work experience 
Expectations of college w< ik 
At t itudes toward col lege 
Attitudes toward self as a learner 
Post col lege aspirat ions 

C. Informal analysis of students in program 

1. From interviews 

2. From teacher's or administrator's impressions 

D. Description of the typical student. This can be constructed by 
combining the modal response to all van ab les "The typical 
student is female, white, 24 yrs. of age, graduated from high 
school, and so on". This is often useful in a summary statement 
of who is in che program. It can be misleading if the student 
body is quite d iverse . 

How doe' the placement program work? 
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This should be a description of the process that occurs for placement. 
Cut-off scores, contingencies, possibilities for exceptions should 
all be described. A flow chart can be useful in describing the system. 
V. What is the nature of the curriculum? 

A. Syllabi can be presented in appendices 
b. What training do faculty receive? 
C. Do ail faculty use the same materials? 
0. How is continuity across sections of a course assured? 
E. How is continuity across levels of a subject area assured? 
VI. How is the program currently perceived? 

A. By students 

1. Remedial 

2. Non- remedial 

B. By faculty 

1. Regular 

Not teaching basic skills 
Teaching basic skills 

2. Basic Skills 

3. Coadjutants 

Obviously much of the analysis called for here requires planning, the 
development of questionnaires, and interviews with faculty and students. 
Several questionnaires appear in Appendix B; basically, these do not require 
a high level of sophistication. The biggest problem with collecting data of 
this t>t>e is to isemble in one place all the various sources of information 
that might be gathered on a student. Information management is a complex 
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isaue and well beyond the scope of this document. It is essential, however, 
that .the record for a student be kept at a central location and that 
information can be added to that record. The easiest way to accomplish this 
is through the assignment of identification numbers which can be placed on 
all questionnaires, tests, and so on. This makes data additions particularly 
easy to do on the computer. 

Appendix A provides some examples of the procedure mentioned here. 
Specifically, there is a history of a college's remedial program (one that 
was evaluated as part of this study); a documentation of a program (at 
Burlington County Community College in New Jersey); and several flow charts 
of placement procedures (at Camden County Community College in New .Wsey). 
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PLACEMENT EFFECTIVENESS 
The placement of students with special educational needs into the 
proper level of remedial work has long been a problem known to educators. 
Binet (1905) was commissioned by the City of Paris to develop a test to help 
place pupils into special schools. His work developed into the Binet-Simon 
scales which later became the Stanfoird-Binet IQ test which is currently so 
popular and controversial. The problems of placing students in programs 

4 

remain today. 

"\ Two basic questions need to be asked in evaluating a placement system: 
1. Are the students being properly placed? 

I. Can the placement program be made more efficient/effective/humane? 

Several'sub-quest ions implied in these two general questions will be 
addressed as we look at methods for answering the questions. 
Are Students Being Properly Placed ? 

There is a natural interaction between the placement of students into 
remedial programs and the nature and content of those programs. For 
purposes of discussion, let us assume that the remedial program itself is a 
satisfactory one. That is to say that the contention "Students are properly 
placed here, but his course ought to be easier.", is logically one which 
$hould be addressed in the next section: "Remedial Program Effectiveness." 

To begin an evaluation of the placement program, a first question to 
ask would be "Are the data that are used in placement decisions valid?" 
Mo*t colleges use standardized tests in making placement decisions. Validity 
evidence for such tests can be found in one of three sources: (1) the 
technical manual provided by the publisher, (2) the Eighth Mental Measurements 
Yearbook (Buros, 1978), (3) journals such as Educational and Psychological 
Measurement or the Journal of Educational Measurement. 
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La New Jersey, students in public institutions are placed into remedial 
programs based in part or in full by their performance on the New Jersey 
College Basic Skills Placement Test (NJCBSPT). If your institution uses the 
NJCBSPT, then the "validity of the data question" can be addressed by 
referring to '/Validation of the New Jersey College Basic Skills Placement 
Tests/ 1 Educational Testing Service (1980, prepared by L. Hecht). This 
report concludes that the test in fact produces valid data for placement 
decisions; the supporting data for this conclusion are fairly strong. If 
your institution use 9 data from sources other than a standardized instrument , 

N 

N 

then the Hecht report would serve as an excellent model for generating such 
validity evidence. \ 

Once the question of the validity of the data being used in placetrtent 
decisions has been established, then one must turn to the question "Are 

\ 

these data being used in a reasonable fashion?" In placing a student 

into a course, there are three possible occurrences: 

1.. The student is placed into a course of proper difficulty. 

2. The student is placed into a course that is too difficult. 

3. The student is placed into a course that is too easy. 

The simplest method of ascertaining which of these three states exists 
for a given student is to ask the student and his/her instructor. If you 
are interested in the reactions of students who have been placed this 
semester by your system, thie might be done at the end of the current 
semester, (if you have a considerable number of drops and adds, you 
might want to obtain these data during say, the second week of instruction.) 
If you are interested in the overall impression of your students concerning 
basic skills placement, a more general survey could be mailed to a sample of 
students or administered in a sample of classes. 

lit 
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Gathering "student/ facul ty satisfaction with placement" data is a 
fairly straightforward task. In Appendix B are copies of two questionnaires 
used in our studies with students. The first is a one-page instr intent for 
use in particular courses The second is a three-page instrument that could 
be administered at any time to a sample of students. (Feel free to use them 
as is or modify them in any manner you wish.) 

Drakulich (1980) has developed an extensive ani quite interesting 
questionnaire concerning placement decisions and the NJCBSPT as part of his 
doctoral dissertation. This questionnaire (which also appears in Appendix 
B) is much more extensive than those we used (and covers a broader area). 
We recommend that the reader spend some time examining this questionnaire. 

Although student opinions of placement provide important information 
concerning the appropriateness of the placement , student opinions suffer 
from bias as well as honest misperception. Faculty evaluation of student 
placement is much more likely to be an accurate reflection of placement 
effectiveness. A questionnaire to ascertain faculty evaluation of placement 
accuracy developed for the Hecht ETS study mentioned earlier is contained in 
Appendix B. As can be seen, the quest ionnaire is very simple in design; it 
can be administered near the beginning or the end of a semester. More 
detailed questionnaires could be developed to ascertain faculty opinion on a 
variety of issues concerning students in basic skills programs, but if the 
issue at hand is primarily one of determining appropriateness of placement, 
the questionnaire as presented is probably quite sufficient . 
Analyzing and Presenting the Data 

Once data on student and faculty satisfaction with placement have been 
collected, the analysis and presentation of the data is not difficult (but 
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it is somewhat tedious). Even for fairly simple analyses with relatively 
small data sets, we recommend using a "canned" computer package such as 
SPSS, BMD, or SAS. Most college computer centers will have at least one of 
these packages available. If the person doing the analysis is a newcomer to 
canned programs, we recommend using SPSS because of the clarity of the 
user's manual. If the person doing the analysis is sophisticated in such 
activities, he/she will probably' want to use SAS. 

In order to present the results of the questionnaires discussed or 
similar questionnaires, frequencies, means, and mean percentages are 
sufficient. Tables 1 and 2 were constructed to present some results of the 
questionnaires discussed. They are self-explanatory and can be readily 
comprehended without a high level of statistical sophistication. 

In terms of presenting material to readers who will not spend a great 
deal of time on the report, graphic displays of data are often effective. 
They are also somewhat more dramatic than tables and have received consider- 
able use in popular news media. Graphs are easy to construct; they simply 
require a translation of the data. For example, in Figure 1, a graphic 
representation of the M Sumnar> of Responses to Instructor Satisfaction 
Questionnaire at College B H is presented. 

For most readers, this graph makes the placement data reflect more 
positively on the placement system than the table does. Interestingly, the 
graph makes placement into Reading 090 look almost perfect and placement in 
Writing U90 weak by comparison. What does not appear in the graph but does 
appear in the table is the sample size tor these groups . The 18.4 percent 
of the students which are reported to be underprepared in Writing 090 is 
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Table 1 

Summary of Responses to Instructor Satisfaction 
Questionnaire at College B 



Number of Number of 

Course Sections Students Percent Placed Percent 

Responding Represented Underprepared Correctly Over prepared 

Writing 090 2 38 18.4 71.1 10.5 

Writing 103 21 383 9.1 88.3 2.6 

Reading 090 2 43 0,0 97.7 2.3 

Reading 102 12 213 9.4 76.5 14.1 

Math 090 21 387 20.4 76.5 3.1 



TABLE 2 



PERCENT DISTRIBUTIONS OF RESPONSES TO STATEMENTS RELATED TO 
STUDENT OPINION OF THE NJCBSPT 



Statement Besoonses 



SA A U D SD NR 

N % N% r % N% N% N % 



N=375 

Overall, I believe the NJCBSPT 103 27.5 175 46.7 50 13.3 28 7.5 17 4.5 2 0.5 

was fair, (l) 

Overall, I believe the test was easy. (2) 31 8.3 94 25.1 79 21.1 118 31.5 46 12. 3 7 1.9 

I tried to do as well as I could on 180 48.0 153 40.8 1 4.5 18 4.8 1 0. 3 6 1.6 

the NJCBSPT. (3) 

Overall, I believe the test was 111 29.6 116 30.9 45 12.0 69 M.4 29 7. 7 5 1.3 

too long (3-1/2 - 4 hours). (4) 

The 20 minute essay was a good 78 2 0.8 144 38.4 47 12.5 66 17.6 36 9 6 4 11 

measure of my ability to write. (5) 

I guessed on most of the items on the 34 9.1 110 2*. 3 79 21.1 113 30.1 38 10.1 1 0.3 

NJCBSPT se-.tence structure test. (6) 

I was so nervous I could not concen- 78 20.8 99 26.4 41 10.9 111 20.6 42 *T. 2 4 11 

trate on the test. (7) 

The NJCBSPT was designed for students 25 6.7 61 16.3 52 13.9 144 38.4 88 23.5 5 1.3 

who are smirtcr than I am. (8) 

It is a good idea to require students 152 40.5 137 36.5 38 10.1 30 8.0 16 4.3 2 0.5 

to take the NJCBSPT. (9) 

I think the NJCBSPT was a waste of 25 6.7 45 12.0 54 14.4 138 36.8 104 27.7 9 2.4 

time. (10) 

The NJCBSPT provided important in- 120 32.0 149 39.7 38 10.1 46 12.3 17 4.5 5 1.3 

formation for my adviser to use in sel- 
ecting appropriate courses for me. (11) 

The NJCBSPT results were a good in- 70 lfl.7 119 31.7 79 21.1 67 17.9 30 8.0 10 2.7 

dicator of my scholastic potent ial . (12) 

9 
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only seven students, and only two sections responded. (Two sections re- 
sponding is also the case for Reading 090.) Although this information could 
have been added to the graph, it is still less likely to be attended to in 
the presence of more visually appealing stimuli. 
Interpreting the Analysis 

One of the largest problems in program evaluation is misinterpretation 
of the analyses that are performed. Let's stay with the table and graph 
that were discussed in the section above and use them as an example to 
provide some general rules for interpretation. 

1. The first thing to do in interpreting data is to get a general picture 
of what's happening. This will help keep things in perspective. In 
the faculty satisfaction data presented here, the overall picture is a 
positive one. For all courses, at least 2/3 of the students are 
(reported to be) properly placed. For all but Writing 090, at least 
3/4 of the students are properly placed (according to faculty). 

2. The next thing that might be examined is those aspects of the data 
which stand out. Probably the two pieces of data which are the least 
consistent with the overall pattern here are the very high satisfaction 
rate in Reading 090 (97%) and the large number of reportedly under- 
prepared students in Math 090 (20. 4%), especially when compared to the 
smal I number reportedly over pre pared students (3.1%). 

3. With a general picture of the data as well as those aspects which stand 
out, the next step is to draw some tentative conclusions concerning the 
placement program. Again, overall the picture is quite good. In math, 
however, we see that a fairly substantial number of students are 
underprepared according to their instructors. If we multiply the 20.4% 
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by the number of atudents who were in the sample, we find that 79 
students were classified as under prepared . This suggests that the math 
remedial sequence might be broken into a two course sequence rather 
than a single course. This is a finding that does not in and of itself 
merit a change in the program, but does merit further investigation 
(What do the math faculty think? What do students think? If we take 
the bottom fifth of the students (based on placement test scores), how 
%e\\ do they perform in Math 090?) It is interesting to note that this 
is actually a concern of the remedial program, not the placement program 
(there is currently no lower course in which to place these students). 

In reading, the placement into Reading 090 looks excellent; 
the placement into Reading 102 is also very good, but perhaps the 
cut-off score for placing out of Reading 102 mi^ht be dropped a point 
or so. It would be worthwhile to find some of the highest scoring 
students who placed in Reading 102 from this year and previous years, 
some of the lowest scoring students who placed out of Reading 102, and 
interview them concerning their regular academic course success. This 
would allow for an informal evaluation of the benefit of Reading 102 to 
those students who are "right on the line. 11 

In writing, there appears to be a problem with students being 
underprepared for Writing 090. The sample is small however, and should 
De regarded with some caution. There are only seven students in this 
M underprepared M group. It is difficult to infer too much from the 
small sample, especially when placement in the higher writing course 
(Writing 103) looks excellent. 
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Two principles emerge from this section on interpreting the analyses: 

1. Results should be explainable. Something caused the data to be the way 
they are. The task it to find the most parsimonious explanation. 

2. These data are not th> only data available. Interp .cations should 
"fit" the data from all sources. Thq conclusions we draw from these 
data have to be consonant with the rest of the data. 

Can the Program Be More Ef ficient /Effect ive/Humane ? 

For most placement programs that have been in operation for several 

years, the overall efficacy of the system is probably quite acceptable 

(severe placement problems are usually apparent). This situation can lead 

to the impression that the placement program is satisfactory in all respects 

and would not benefit from some fine tuning. In this portion of the PlaCe- 
cJ 

ment Effectiveness section, we will examine evaluation techniques of a more 
formative nature; that isj how can a system which is considered good be made 
better? 

Analysis of the Program 

To begin, a critical analysis of the program itself can be made. 
Although this analysis could be performed by program personnel, an individual 
who is familiar with basic skill" programs, but external to the program in 
question might be helpful. For example, directors of basic skills programs 
in neighboring colleges might - gree to examine one another's programs. 

The analysis should consist of a step-by-step examination of the 
placement procedures. The analysis might be broken down as follows: 
Ik Identification and notification of students concerning the testing. 

How are students identified? 

How are students notified of the testing? If by letter, is the letter 
bureaucratic or friendly? 



-21- 

How many testing dates are there? Do itudenti have an option? 

2. Nature of the testing. 

Under what conditions does the testing take place? 

Does anything but testing occur on the day of the testing? Is there 

any consideration of a social aspect to the day? 
What about no-shows? How are they identified, contacted, tested? 

3. Nature of the placement/counseling. 

Who auMyzes the scores for placement recommendations? 

How rigid is placement with respect to scores? Does any latitude 

exist? Can counselors/students override the scores? How? 
Who explains the scores/remedial courses to students? 
What consideration of student affective response to placement is made? 
Do students have any recourse on placement decisions? Can the test be 

retaken? 

Of course, other aspects and questions will be peculiar to each placement 
program. The list above is intended as an example of an approach to analysing 
programs . 

Analysis of Record 

Some evidence reflecting on the placement pro 6 ram is collected routinely 
by the college. In particular, this would be the number of students who 
transferred from one course to another or who dropped courses altogether 
For example, should you find that there are large numbers of students 
transferring into the more elementary remedial courses, this suggests 
raising the cut-off score for the higher elementary course. Transfer data 
would be particularly appropriate for those institutions who use some form 
of secondary evaluation in remedial classes once students have been placed. 

° / 
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We have found that actually referring to the college drop/add data can 
be quite difficult and confusing. It may be easier to survey faculty (even 
\nformally) about the magnitude and direction of transfer activity in their 
courses • 

In Appendix B, there is a document concerning students who were retested 
at the beginning ot a remedial course at Burlington County College. This is 
a good example of th2 type of information concerning the placement* system 
that can be gathered by looking at the "running record" of student performance. 
Interview Data 

Some information comes out only in the course of conversation. Especially 
in terms of making placement a more humane activity, interviews are extremely 
valuable. In terms of examining the placement programs, the people who 
could provide the best formative information would be students and the 
individuals who counsel students on placement. 

In our evaluation efforts, we found gathering a small group (4-10) 
of students ^nd posing some general questions concerning placement to be 
quite effective. Common experiences and reasonable recommendations came 
forth from the 1 group. As a side note, the students at both colleges we 
worked with felt that the interview session itself was a valuable activity. 
They expressed the opinion that regular meetings of students (sometimes 
with, sometimes without staff) to discuss the placement and (especially) the 
remedial program would be beneficial. 

Questions used in our student interviews are listed below: 
1. (Interviewer traces the placement procedures showing samples of documents - 

letters to students, etc. - where appropriate.) How accurately does 

the procedure described and the material presented match the method in 

which you were actually placed? 
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2. How fair and equitable do you Chink Chit procedure is? 

3. What was your initial reaction to being placed in a Basic Skills 

Program? 

4. How has your attitude toward the program changed? 

5. Do you feel that you were "placed" properly? 

6. Do you feel you have any channels of communication through which to: 

a. Gain an understanding of why you were "placed" where you were? 

b. Appeal what you may perceive is an improper placement? 

c. Air any grievances you might have? 

We employed the small group method in our faculty interviews also. 
In these groups were faculty who were: 

1. Full-time remedial faculty. 

2. Regular academic faculty who taught one or two remedial courses. 

3. Co-adjutant faculty who taught only remedial courses (usually on a 
part-time basis). 

These interviews were not as successful as we might have hoped. Loyalties 
to one's group (as listed above) inhibited the free exchange of ideas and 
opinions. We would recoramendogathering faculty together by group status and 
not across group* (or at any rate, not just across groups). Below is a list 
of questions we used in the faculty interviews; 

1. (Interviewer traces the placement procedure.) How accurately does the 
procedure just described match the actual placement of students into 
the program? 

2. How adequate and appropriate do you view the placement procedure? 

3. Would you comment on your satisfact ion/dissat isfcct ion with your role 
in the placement procedure? 
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4. What impact have the basic skills placement and remedial programs had on 
your college? 

5. What do you perceive the role of the placement and remedial programs to be? 

\ 

6. Is there an apparent need to develop any additional or improved evaluation 
procedures to assess the effectiveness of the placement and remedial 
programs? 

7. What kinds of feedback have you received from students concerning the 
effectiveness of th? placement aid remedial programs? 

In our evaluation, we also interviewed administrators of remedial 
programs and administ rator* of colleges. These interviews were essential 
because of our "outside" status and a need to assess institutional support. 
These interviews would not typically be part of an "in-house" evaluation. 
Summary 

This section has relied heavily upon student and instructor satisfaction 
as indicators of a successful placement program. In the minds of some 
evaluators, use of these indicators may raise a question of the validity of 
the data being used. Our rationale for focusing on satisfaction data was 
twofold: x 

1. If there are serious problems with your placement system, you won't 

need a sophisticated mechanism to pick up that fact; people will beat 

\ 

down your door telling you about it. \ 

2. If you are interested in researching various alternative selection 

\ 

systems, there is a section on that undeV the Special Studies subheading. 

\ 

In summary, the guiding rule to assessing \placement effectiveness 

\ 

involves answering the questions: 
1. How can we document the effectiveness of the program as it exists? 
1. How can we shed light on areas for improvement t 
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PROGRAM EFFECTIVENESS 
The evaluation of remedial programs at any level of education poses a 
particularly difficult task. In addition to the difficulties encountered in 
any evaluative activity, remedial programs usually have three additional 
features which exacerbate the problem: 

1. All students identified as candidates for the remediation must be 
included in the remediation. (Thus we have no control group.) 

2. Candidates represent one end of the ability continuum; suitable contrast 
subjects are not available. (Thus an^ kind of comparison is difficult.) 

3. Mortality rate (in the statistical sense) is very high. (Thus we can 
only know that a treatment works if people don't drop out.) 

These three problems exist in varying degrees in public colleges and 
pose problems for evaluation of remedial programs in those colleges. 
Remedial programs also exist in the public elementary and secondary schools, 
most notably under the ESEA Title I provisions for economically disadvantaged 
children. The federal government has developed evaluation models for those 
programs which attempt to ameliorate the problems listed. We have examined 
the applicability of those models for college basic skills remedial programs 
and concluded that none of them are directly applicable. These models are 
discussed further in the section on Special Studies. What j£ contained in 
this section is a variety of evaluative techniques which can be used for 
remedial program evaluation. None of them are faultless. What we have done 
is to list for each of the seven techniques to be presented: (1) "Pros": 
benefits of the technique, (2) "Cons": problems of the technique, (3) 
"Summary of Pros and Cons," and (4) "Method": how to employ the technique. 
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We are not recommending that you use all of them. They are presented as 
options which may be more or less useful in your evaluation efforts. 

Before presenting these techniques, we need to attempt some common -J 
understanding of remedial programs. We are making the following assumptions 
in discussing these techniques: 

1. That the primary goal of remedial programs is to prepare students for 
regular college work. 

2. That this goal can b* reached in a variety of ways, but tha* the predomi- 
nant methods are through remedial courses\in reading, mathematics, and 
writing, and tutor ials/iabs which may accompany these courses or operate 
independently of them. \ 

The seven evaluative tecV^ques which make up the remainder of this 
section are: \ 

1. Pre-/Posttesting 

2. Success in Future Work \ 

3. Success in Remedial Courses x 

\ 

4. Faculty Satisfaction 

5. Student Satisfaction \ 

6. Analysis of the Curriculum 

7. Evaluation of J,abs/Tutor ials 
1. Pre-/Posttesting 

Since in most instances students are pretested using a standardized 
instrument, there would be a natural inclination to posttest with the same 
instrument and compare the pre-/post results. 

Pros - Perhaps the strongest argument for pre-/posttest ing using a 
standardized test is that the test represents the best efforts of professional 
psychometricians in determining what college-relevant basic skills are. If 

33 
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students are shown to perform substantially better on the posttest 

at opposed to the pretest, then it would only be reasonable to conclude that 

the remedial program was effective. 

A second strong point of this technique is thst the results sre 
readily interpretable snd coranunicable. If a student's performance on the 

posttest is sufficient for him/her to have been initially "placed out" of 

v" 

the remedial cours\he/she has just completed, this is compelling evidence 
of effective remediation and easily understood even by those who are 

statistically naive. 

A third strong point of this technique is that the data are 
suitable for a wide variety of statistical analyses, including analysis of 
variance and multiple regression. 

Cons . There are some serious problems with pre-/posttesting that 
need to be considered. The first (and perhaps largest) problem concerns the 
motivation of students engaged in the testing. During the pretest, students 
are highly motivated, since good performance can place them out of remedial 
courses. If the placement is given as a posttest, there is almost no 
motivation to perform well whatsoever, unless the test is used as part of a 
student's grade in the course. In an evaluation we conducted as part of 
this study, we found students whose posttest scores dropped by as much as 30 
points from the pretest in a college that did not use the posttest as part 
of the grade. Obviously students do not regress by 30 points as the 
result of a semester's instruction. Cases such as those are easy to spot 
and take into consideration in an analysis, but what about scores that drop 
5 or 10 points or show little or no gain? How accurately haj growth been 
reflected by those scores? It is impossible to determine. This problem is 
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quite serious and needs to be addressed before putting too much weight 
behind a pre-/posttest analysis. 

If the post test is used as part of the grade in a remedial course 
(or as the exclusive determiner of advancement in the sequence), then the 
motivation problem becomes much less severe. (Of course, if the posttest is 
10-20Z of a grade and if a student is doing quite well going into the test, 
this could still be a problem.) There are, of course, problems incumbent 
with having the test be part of the grade. Anxiety toward the test may 
impede learning; teaching to the test may occur; security of the test could 
be a problem, and so on. These issues would have to be considered if 
mandatory, grade-related posttesting were to be implemented. 

A second problem area concerning posttesting concerns the transfer 
of skills from the remedial course to a regular college course. Joy Hughes 
from Burl* igton County College reports that students who perform well using 
the NJCBSPT still show difficulty (in some cases) in regulat course work 
because of dif f icr.I ties in transferring skills from remedial to regular 
courses. This problem will be discussed further later, but the point here 
is that the posttest may not be capturing exactly what you want students to 
be able to do (i.e., succeed in regular courses). 

A third problem with pre-/posttesting is a purely statistical one. 
The problem is called the "regression effect." Stated simply, the regression 
effect occurs when the bottom portion of a distribution of scores that have 
measurement error is isolated and then retested. Even if the retesting 
occurs the next day, the scores will tend to rise. This is because the 
people in the bottom portion of a distribution are more likely to have had a 
negative (or depressing) error made on their pretest measurement. When 
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retested, as a group, their errors would tend to even out (sum to zero). 
Thus the post test mean score would be higher than the pretest mean score. 
Just how much higher the posttest mean would be is a function of the relia- 
bility of the measure in question. 

The perceptive reader will note that the regression effect works in 
favor of the evaluation. Actually, if a pretest is given at the beginning 
of the remedial course (not the placement instrument, but a new testing) 
and again at the end of the course, the regression effect is eliminated. 

Sunmary of Pros and Cons . Pre/Post is not the sine qua non of 
evaluation (or of anything else, for that matter). It can however, provide 
good evaluation evidence if used and interpreted properly. The regression 
effect problem is one that could be ameliorated if the motivational problem 
could be overcome. If you believe the motivational problem could be minimized 
at your institution, then you may well want to use pre-/posttest ing. We 
would recommend using an instrument other than the placement instrument but 
if you do use a placement test/posttest model, recognize that sme of the 
growth noted will be due to regression (just how much is impossible to tell 
without knowing what proportion of the students were placed in the remedial 
program as well as the reliability of the test f your students.) 

Method. There are a variety of techniques for analyzing and displaying 
the results of a pre-/posttest analysis. Basically, the data used in this 
method are the pre- and posttest scores. If you use different forms of a 
standardized instrument in pre- and posttesting make sure you always use the 
scaled scores provided by the publisher and not the raw scores. Raw scores 
from different forms of the test are not comparable . Make sure that the 
scores are the relevant scores (don't use reading comprehension for your 
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remedial math classes). Also, if you use holistically scored essays, both 
pre- and post- essay scores must come from the same scoring session. 

The basic statistical tool used in analyzing pre-/posttest data is the 
dependent samples (sometimes called "correlated samples") t-test. This 
procedure can be found in almost any elementary statistics text. This 
analysis could be performed on any breakdown of students as well as all 
students (in a course) combined. For example, you could show results as in 
the following (simulated) table: 



Gain in Reading Comprehension Scores* 





All Students 
in Reading 
Remedial 
Courses 


All 
Students 

in 101 
Sect ions 


All 
Students 

in 102 
Sect ions 


Students 

in 
101:235 


Students 
in 

101:238, etc. 


Mean Pre 


56.5 


52.7 


59.4 


54.1 


53.9 


SD 


8.4 


8.8 


7.6 


9.2 


7.8 


Mean Post 


62.4 


58.0 


66.3 


58.6 


56.2 


SD 


11. 1 


10.6 


9.2 


12.8 


8.6 


Mean Gain 


5.9 


5.3 


6.9 


5.5 


3.2 


t statistic 


8.3 


6.4 


10.4 


2.3 


1.6 


P-value 


.001 


.001 


.001 


.01 


.08 


N 


430 


248 


182 


24 


20 


♦These values 
scores . 


have been generated as an example. There 


are not actual 


It is easy to become overly impressed by the 


results of 


such an 


analysis. One 


should keep in 


mind that the 


t-test is s 


imply establishing 


that the students learned more 


than nothing at all! An 


instruct ional 



intervention should be better than nothing. "Significance" in a statistical 
sense should not be confused with "importance" in an educational sense. 
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A second problem with Che t-test is thst it does not provide s 
picture of the oversll performance of a course or a section of a course. 
Several methods might be used to convey these. The first is called the stem 
and leaf diagram. The stem and leaf diagram allows for a picture of the 
data to be seen without losing the actual scores. Below is an example of a 
stem and leaf diagram for a single group of scores: 

Stem and Leaf Diagram of Single 
Set of Scores 



8 

0 3 

5 5 6 9 

0 0 1 2 2 4 

5 6 6 6 7 

0 0 13 

7 

0 3 



7 
7 
6 
6 
5 
5 
4 
4 
3 
3 

Basically, the tens column is represented on the left of the "stem" and the 
ones are represented on the right of the stem. In the example above, there 
was one score of 35, one score of 40, 43, 47; two scores of 50, and so on. 
The tens column has been broken in half (two 3's, two 4's, etc.) in order to 
spread the scores out more. Scores from 30-34 would be placed next to 
the lower 3 in the tens column; scores from 35-39 in the upper 3. 

Once the basic stem and leaf idea is perceived, a variety of 
elaborations might be employed. First, pretest scores could be placed to 
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the left of the tens columns and the cui off score for testing out of the 
course might be identified. This would suit in a stem and leaf diagram 
like the one below: 

Stem and Leaf Diagram for 
Reading 090 Section 123 



Pretest 



Posttest 



a 

4 4 3 0 0 
9 8 6 6 
3 : 0 
8 5 5 



7 
7 
6 
6 
5 
5 
4 
4 
3 
3 



0 0 1 

T) 6 8 8 8 9 

0 4 4 
7 9 

-cut -off score 
for Reading 090*55 



Not only can w? get a general picture of performance using the stem and leaf 
diagram, but all of the individual scores are retained. Stem and leaf diagrams 
can be constructed for courses, sections, or other relevant breakdowns. 

Another graphic display technique and another tabular display technique 
for showing gains in pre-/post test ing are presented in Appendix C. 
2. Success of Remedial Students in Future Work 

A second evaluative technique is to look at how well formerly remedial 
students perform in regular college work. It should be recognized at the 
outset that this technique will not provide information on your remedial 
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program as it exists today , but rather as it existed when your "formerly 11 
remedial students were in remedial courses. 

Pros . In some respects this is the single best evaluative technique 
available. It is the most direct measure of the goal of the remedial 
program: to prepare students for regular college work. If variables such as 
grade point average, graduation rate, and so on, are comparable for formerly 
remedial and non-formerly remedial students, then the efficacy of the 
remedial program is rlear. 

Cons. There are several problems associated with comparing formerly 
remedial and non-formerly remedial students: 

1. The comparison group (non-formerly remedial students) presents a fairly 
high standard to attain. Just how close to that standard would be 
considered acceptable is an arbitrary judgment. If the GPA's of the two 
groups were not statistically significantly different, this would be 
strong evidence of the quality of the program. Given that such a standard 
is close to unattainable, the question becomes: M How close will we accept?" 

2. The data necessary for such a comparison are exceedingly difficult and 
tedious to obtain unless you have a fairly sophisticated management 
information system. 

3. The comparison relies on course grades, which are frequently unreliable, 
and graduation (drop out) rates, which involve a number of variables 
unrelated to academic preparedness. 

Summary of Pros and Cons . If you can get these data, and you are 
interested in evaluating your program as it has existed over the past two to 
three years, this is the best technique of the seven. The only major problem 
is that your comparison group is not; students who needed remediation but didn't 
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receive it, but students who didn't need remediation. Thus, the benchmark 
is high; if you approach it, you probably have an excellent program. 

Before proceeding to the method of analysis, a comment on differ- 
ences between two-year and four-year colleges should be made. The express 
purpose of many two-year schools is to offer the opportunity of college to a 
diverse population. This inherently necessitates admiting a number of "high 
risk" students. Consequently, the proportion of students who go on to 
complete remedial work and obtain college-level credit is often low. The 
evaluation question is "Given the nature of the admitted students and the 
degree of their remedial needs, are a t-tisfactory number of students 
acquiring college credit?" The issue: "How large a proportion is satisfac- 
tory?" is particular^ Tficult. We can imagine situations where if 15% of 
entering remedial students receive regular college coursework credit, a posi- 
tive assessment of the program would result. It depends on what population 
the college is intending to serve. 

For four-year colleges, this is frequently much less of a problem. The 
point here is that in -der to set a reasonable standard ,* one needs to 
consider the goals of the program. 

Method of Analysis . In order to make these comparisons, grade 
point averages and/or dropout (persistence) data are necessary for samples 
of formerly remedial ©students. For comparing CPA's, an independent samples 
t-test can be computed. If you have multiple remedial groups (for example, 
students who took one remedial course, two courses, and so on), a one-way 
analysis of variance cbuld be used. Do not, however,, be overly impressed by 
the hypotheses tests. What is important to look ay is the mean CPA's for K 
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the groups. Whether they are clot* or not can be assessed by simple inspec- 
tion. Results might be presented in tabular form as follows: 



No 

Remedial 
Courses 



One 
Remedial 
Course 



Two 
Remedial 
Courses 



Mean GPA 

after four 2.45 
semesters 



2.36 



2.15 



SD 
N 



.34 



39 



.46 



643 



337 



259 



At the community college level, GPA is often a less meaningful 
statistic than, say, number of credits completed. A comparison using number 
of credits completed can be made by using the same methods as above, but 
simply changing the outcome variable. This could also be displayed graphically, 

If you are interested in comparing dropout rates, then the simple 
percentage of dropouts for remedial versus non-remedial students can be 
compared. The statistical significance can be assessed by using the chi- 
square statistic and the table below: 



Dropout 



Persi ster 



Remedial 
Non-Remedial 



n 


n 

> 


n 


n 



(Chi-square with one degree of freedom, 
Yates* correction should be applied) 



3. Success in Remedial Courses 



A good technique for formative evaluation is the success that 
students are having in remedial courses. Should your program have positive 



9 

ERLC 

hriiniiiffnrrriaaii 



A 2 

«4 A* 



-36- 

results based on technique 2 (comparing remedial and non-remedial students), 
then this procedure will also provide good information of a summative nature. 
That is, the argument could be made that "Our formerly remedial students are 
doing quite well, and our current remedial students are doing well in 
comparison with their predecessors." 

Pros . This technique is probably most appropriate for internal 
analysis. Questions such as "Are our grading practices consistent?"* "How 
do regular faculty compare to basic skills faculty compare to coadjutant 
faculty in grading pract ices?", "What do dropout rates look like by course 
and section?" can be addressed with this technique. 

Cons . This technique is not going to be overwhelmingly convincing 
outside of the institution. Grading standards vary from instructor to 
instructor; dropout rates from remedial programs will vary greatly depending 
on the nature of the student body at the institution. Another problem with 
this evaluation technique is that if conducted on a section-by-section basis, 
it could generate substantial n gative faculty reaction. 

Summary of Pros and Cons . If combined with a strong result from 
technique 2, this technique is convincing. In the absence of that result, 
this technique serves primarily internal, formative needs. 

Method of Analysis . The best analysis method here would be a 
simple cross-tabulation of grades (or dropout rates) by course, section or 
other relevant breakdown as in the table below: 
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Section 



A't 



B't C's 



D's 



F'a 



Withdrew 



090:101 3U5Z) 4(20Z) 12(60Z) 1(5Z) 0(0Z) 0(0Z) 

s 

090: 102 6(60Z) 2(20Z) 0(0Z) 0(0Z) 2(20Z) 0(0Z) 

090:103 2(8Z) 5(20Z) 7(28Z)3(12Z) 3(12Z) 5(20Z) 

110:101 2(20Z) 3(30Z) KiOZ) K10Z) K10Z) 2(20Z) 



4, fa culty Satisfaction 

Another good method of internally monitoring the remedial program ■ 
is to assess faculty satisfaction with the program. This can be done with a 
survey instrument, through interviews, or through a study, where regular 
faculty are asked to rate a small sample of their students, some of whom are 
formerly remedial, some of whom are not (more on this later). 

Pros , Assessment of faculty satisfaction with the remedial 
program allows for early identification of trouble spots in the program and 
also engenders a sense of participation on the part of the faculty. Further- 
more, since the faculty are the individuals who work with the "products" of 
the remedial program, there is a sense of ecological validity to their 
opinions. That is, if the English faculty is satisfied with the students 1 
abilities following remediation, then the remediation must be successful. 

Of course, "faculty 11 does not mean just those individuals who 
teach regular college courses, but also includes the faculty who teach 
remedial courses* Thus, faculty satisfaction has two rather distinct 
connotations. The satisfaction of the remedial faculty with the remedial 
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program reflects the professional judgment of those individuals involved 
in the process, and hence is valuable. 

Cons . From an external perspective, there is a certain ldpk of 
rigor in asking those people who are performing a task if they think they 
are doing it well. The opinions of the remedial faculty are thus of primary 
importance to an internal perspective. The opinions of regular faculty 
concerning the remedial program provide more convincing data but are difficu 
to compare from setting to setting. That is, 11 "If faculty at College A are 
pleased with the remedial efforts of their basic skills program, and the 
faculty at College B are not pleased, can we conclude that the basic skills 
program at College A is superior to College B?" 

Summary of Pros and Cons . The assessment of regular faculty 
opinion of formerly remedial students is a good method of ascertaining 
program efficacy. It is rather difficult to compare ratings across colleges 
thus, such ratings do suffer somewhat from lack of interpretability. The 
opinions of the remedial faculty are an excellent means for the early 
identification of problems, but are not particularly convincing data from an 
external perspective • 

Method of Analysis . There are three possibilities here. The 
first is an interview of faculty. This is the method we used in our evalua- 
tive efforts* We interviewed faculty in small groups; the groups contained 
regular faculty, fulltime remedial faculty, and coadjutant remedial faculty. 
As mentioned in the section on Placement Effectiveness, we do not recommend 
this as the exclusive procedure for faculty interview. Instead, we would 
recommend gathering faculty in small groups that were homogeneous with 
respect to their position within the college. After those interviews have 
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been conducted, perhaps an interview which cuts across the various types of 
faculty would be productive. Below is a list of questions that we used in 
our interviews: 

1. What impact have the basic skills placement and remedial programs had on 
your college? 

2. What do you perceive the role of the placement and remedial programs to 
be? 

3. Is there an apparent need to develop any additional or improved evalua- 
tion procedures to assess the effectiveness of the^ placement and 
remedial programs? 

4. What kinds of feedback have you received from students concerning the 
effectiveness of the placement and remedial programs? 

5. What do you see as legitimate concerns of students enrolled in the 
placement and remedial programs? 

b. Do you believe there exist adequate channels of communication for airing 

of concerns which you might have with the program? 
7. How would you rate the basic skills courses in terms of: course 

relevance, skill development, course expectations, outcome proficiency, 

and requirements? 

tt. How much latitude do you have with respect to implementing your own 
instructional strategies and techniques? 

Has the college provided/sufficient safeguards to ap'ure the integrity 

\ -/ ■ /• 

\ and purpose of the basic skill's placement an remedial programs for the 

"• r 

students'? \ 

\ 

10. How\ould you assess the iitfpact ot\ the placement and remedial program 

on stu^nt academic success in regular college English and math courses? 
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11. How would you evaluate the placement and remedial programs at this 
college? 

12. What suggestions do you have for program improvement? 

A second method for gathering faculty opinion data is, of course , 
a questionnaire. The questions could cover basically the same areas 
that were covered in the interviews; the format would be different. Also in 
the questionnaire, you would want to find out information concerning status 
in the college, years at the college, courses taught, and so on. 

A third method for assessing faculty satisfaction with the remedial, 
program would be to set up an experimental study concerning formerly remedial 
and formerly non-remedial students. Ten (or so) regular courses could be 
selected and six students identified in each course. The six students would 
be three formerly remedial students and three non-formerly remedial students. 
The faculty w^uld be asked to rate the six students in their class (at the 
end of the course) regarding the students' academic preparedness and perform- 
ance in the class (the students' status would not be made known to the 
faculty). Faculty could rate students on several Likert-type items. These 
ratings could then be used to compute t-tests on the differences between 
remedial and non-remedial students. 

This method is essentially identical to method 3 (Student Success 
in Future Work) except that the ratings would probably be more reliable than 
grades. Data analysis and presentation would be the same as in method 3, 

This third technique requres some work, but would provide particu- 
larly strong data. In most instances, it would be worthwhile if the courses 
and students selected stayed within a subject area. That is, you shouldn't 
use formerly remedial reading students in a study involving 10 sections of 
nath courses (unless the students are also formerly remedial math students). 

47 
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5. Student Satisfaction 

A critical component of evaluating any instructional program is 
the assessment of the students 1 opinions toward the program. 

Pros. Students comprise the group that is most sensitive to the 
actual workings of the remedial program. Student satisfaction data are not 
only excellent for internal analysis, they are also fairly convincing from 
an external perspective. Since being placed into a remedial program is 
inherently negative in its connotation, if students report a positive 
experience in the program this is strong evidence of program effectiveness. 

In the two college remedial programs that we evaluated, as well as 
several other evaluations of colleges that we looked at, student evaluations 
of remedial programs were quite positive. These data might be used in an 
introduction to th£ program for newly placed students. 

We also found that students enjoyed the opportunity to express 
their feelings concerning the remedial program. (This aspect of evaluation 
could actually be incorporated as part of the remedial program.) 

Cons. Again as in the faculty satisfaction section, attitudinal 
data are by their nature subjective. Since the students are the consumers 
of the remedial effort, their opinions are more likely to be taken seriously 
from an external perspective. 

Summary of Pros and Cons . Student satisfaction data are important 
and would be expected to be included in an evaluation of an instructional 
program concerning those students. 

Method of Analysis . Two methods might be employed to assess 
student satisfaction with the remedial program. The first would be to 
interview the students. We found the approach described in the Placement 



43 



-42- 

Ef fectiveness section to be quite useful. The questions that we posed in 
that interview concerning remedial effectiveness were: 
1* Have you found the basic skills courses to be: 

a. Relevant to your needs? 

b. True to course outline? 

c. Reasonable in terms of expectations? 

d. Important to your 3uccess in the next basic skills course in your 
sequence? 

e. Important to /our success in regular college English and math 
courses? 

2. Has the College provided sufficient safeguards to assure the integrity 
and purpose of the basic skills program for the students? 

3. How would you evaluate the placement and remedial programs at this 
College? 

4. How satisfied are you with the services offered by the Placement Office? 

5. What suggest ion/ s would you make for imnproving the program? 

A second method would be to survey student opinions with a question- 
naire. In Appendix C there are several questionnaires which could be used 
to assess student opinion of remedial program and course effectiveness. The 
construction, analysis and presentation of questionnaires would be the same 
as discussed in the section on student satisfaction in Placement Effectiveness. 

6. Analysis of Curriculum 

A method for insuring consistency of instruction as well as for 
improving the match between the remedial program and the needs of students 
in regular courses is a critical analysis of the curriculum. 
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Pros and Cons . A critical analysis of thfe curriculum allows for 
the two positive benefits mentioned above. What it does not provide is 
information concerning whether the curriculum is well-implemented or particu- 
larly effective in real life. Of course, techniques 1-5 address those 
issues. 

Method of Analysis . Analyzing the curriculum in a particular 
subject area would require professionals from that area to generate a list 
of objectives or behaviors that should be included in instruction. It might 
be worthwhile, for example, to have several regular math faculty members 
develop such a list of objectives which should be mastered before entry into 
the first regular math course. First, once the list of objectives exists, 
the curriculum of the m*th sequence can be matched to the objectives to look 
for gaps in the curriculum. Second, several remedial math instructors could 
indicate which objectives were covered in their math classes. In this way, 
the curriculum (as it exists in the mind of the remedial program director) 
can be matched against the needs of students as expressed by regular math 
faculty. Third, the actual math curriculum (as it is taught in classes) can 
be matched against the curricula of the math faculty and the remedial 
program director. 

Analyses «uch as the one descjioed above may well have taken place 
dt the inception of the remedial program. It might be worthwhile to run 
through that process again to check for slippage over the years. 
7. Evaluation of Labs and Tutorials 

Many colleges use lab* or tutorial sessions. These are an importan 
part of the instructional process, but it is rather difficult to assess 
their impact. 
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Pros . Evaluation of labs and tutorials is a necessity. As is the 
case with most things, they can be run poorly or well. Attendance rates and 
student satisfaction are good indicators of the success of labs/tutorial 
sessions. 

Cons . It is almost impossible to assess the cognitive rowth of 
students as a result of their participation in a lab or tutorial session. 
Hence, the data gathered must be attendance data or attitudinal. 

Method of Analysis . Below are examples of the lab/tutorial 
recordkeeping system and evaluations from Camden County College which would 
serve well as an example of how to provide for evaluation of lab/tutorial 
sessions. 



-45- 



Nomc ol 1.4b i m 
Campus : 



MATH LAB - \~ LL J ">J°L 



Date: 12/27 /79 
Special 1st : 



Tabic 
I 


Course 
Name 

DVlC T 

BMS I 


Course 
Name 

o tic T T 


Course 
Name 

PM T 


Course 

Name 
CM II 


Course 
Name 
Other 


TOTAL 




I stuck iUV 


68 / 


50 >^ 




50 j/ 


213 y^ 


ni V 

DAY 


1 7 ^^"^ 

'fvisits 
97 


/ 212 


/ 204 


/ 72 


/ 172 


y' '57 


EVEN INC 
















27 / 


68 


50 / 


18 / 


50 / 


213 y^ 


TOTAL 


s 97 


/ 212 


/ 204 


S 72 


/ 172 


757 



Table 
II 


Course 
Name 
BMS I 


Course 
Name 
BMS II 


Course 
Name 
CM I 


Course 
Name 
CM II 


Course 
Name 
Other 


TOTAL 


1 

visit 


^students/ 
./percent 


30 / 
y / 44% 


17 / 
/ 34% 


6 / 


15 

30% 


78 

y^ \VU 


2-5 

visits 


/^ 48% 


28 / 
/ 4 1% 


24 y 
/ 48% 


/ 44% 


23 

46% 


96 yS 
yS 45% 


6 + 
visits 


4 / 
/ 15% 


10 / 
/ 15% 


9 jT 
/ 18% 


4 / 
/ 11% 


12 / 
/ 24% 


39 

18% 


V>i 

ERjC 


i students 
27 


68 


50 


18 

-53- 


50 


213 

* 



46 



•ocStc. 



LMI 

J. L 



.Ttrrn. 



htltUI 



oursa. 



S«m«tt«r Yr. 
Numbtr / 



istructor. 



fading □ Writing □ Math □ 
tf>ar 



Effort: 



Progress: 



1. ■ adequate 

2. ■ inadequate 

1 . • txcelltnt 3. • minimal 

2. ■ satisfactory 4. « nona 



Recommendation: 1 . - no further tutoring 

2. * additional tutoring 

3. - rtguiar tutoring necessary 

4. - refer to counselor 



TIME 
In Out 



Auutanc* Provided 



ERIC 



53 



-47- 



SPECIAL STUDIES 

As mentioned in the introduction to this document, the ESEA Title 
I evaluation models were considered as possibly being applicable to college 
basic skills remedial programs. After such consideration, it was determined 
that none of those models were directly appropriate for college basic skills 
remedial programs. The models may however, be useful for special studies 
that a college might wish to conduct. In this section, we will briefly 
describe the three Title I models and suggest what utility they may have for 
college remedial programs. 
1. The Treatment/Control Group Design 

This is a classic experimental design in which students are placed 
randomly into a treatment or control group and then tested at the end of the 
remediation to examine treatment/control differences. The probl m w.th this 
design is that it requires placing children who need remediation into a 
"no-remediation" situation which is clearly undesirable. 

There could be a use of this design for college remedial programs. 
If the basic skills office is interested in the efficacy of an alteration of 
the remedial program, or in a new instructional technique, this new program 
can be examined with this model. Students could be randomly assigned to the 
new program (or programs) or to the existing program. At the end of tne 
semester, measures can be taken on achievement or attitude of the students. 
(If students are randomly assigned, no pretesting is necessary.) These <Jdi «i 
are then subjected to a t-test analysis (if there is just one experimental 
treatment), or to one-way analysis of variance (if there is more than one 
treatment). This procedure will provide a rigorous test of a new program. 
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2. The Standardized Control Group Design 

In this design, standardized tests are used to provide a "national 

control group" which is based on the performance of a representative sample 

of children from across the nation. This is clearly not possible in the 

college remedial setting. The closest approximation to it would be to use 

the cut-oft scores for pldcement into remedial sections as a quas i-cont rol 

situation. The basic evaluative argument here would be to show that students 

who have gone through remediation would no longer be placed into that 

itwilial course based on their posttest results. This would allow for a 

tdhte sik 11 as the one below to be constructed. 

% of Reading I Students of Reading U Students 

Who Would Place Into ,;ho Would Pl*.;c into 

Reading II Based on English 101 Based on 

Posttest NJCBSPT Results Posttest NJCBSPT Results 

Section 101 Section 102 Section 201 Section 202 

k 

48X 86% 60% 56% 

. The Regression Discontinuity Design 

After examining the literature on the regression discontinuity 

design and relating it to college remedial programs, the authors felt that 

this design is inappropriate for any kind of college remedial program. It 

is an interesting evaluative idea, but appears to have too many difficulties 

in implementation to be of practical value. 

Other Studies 

There are two other studies that colleges may wish to carry out 
that we will discuss in this section. They are more appropriately labelled 
research than evaiuacion, so we did not discuss them in earlier sections. 
I. Examining Alternative Placement Rules 

Under any placement system, you are going to end up with studenls 
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who barely placed out of a remedial course and students who barely placed 
into it. Since these students were very similar in their academic abilities 
it would be interesting to follow the two groups through your college. This 
could be accomplished by first identifying two such groups, say, 20 students 
who barely placed out of the highest remedial reading course and into the 
first English course, and 20 students who barely missed the first English 
course. The grade point averages, dropout rates and attitudes (academic 
self-concept, feelings toward the college, and so on) can be compared for 
the two groups. 

The real attractiveness of this idea is that you needn't wait two 
or three years for results to come in. You can run this study by going back 
two or three years and identify studep* s who are now in their fourth or 
sixth semester. 

Another idea related to this is to generate several alternative 
placement proc?dures to the ones you have. Apply these procedures to 
students and identify the following groups: 

Old Procedure 
Remedial Not Remedial 

Remedial 

New 

Procedure 

Not 

Remedial 

Again, gather the data mentioned in the earlier discussion and and you can 
make comparisons to answer these questions: 

1. How well do students do who would be remedial under the new system but 
weren't under the old system? 



2. How well do students do who would be remedial under old system but not 

under the new system? 
There are probably other possibilities here also. The basic idea is to use 
the existing data and decisions to examine possible alterations. 
2. Examining Levels of Remedial Identification 

Some colleges do not notify students *of the fact that they need 
helo in basic skills, but do notify the faculty in the courses they take of 
their needs. The idea is to have the faculty provide help to the students > 
without having students feel that they are second-class citizens. 

This is an intriguing idea and there are many possible variations 

of it : 

1. Students could be assigned volunteer peer tutors such that only the 

r 

tutor and the student knew that students were in need of remedial help, 

2. Peer tutors and faculty (as described above) could be m^de cognizant of 
students' remedial needs. 

3. Students who have remedial needs could be informed of those needs and 
then offered a variety of remedies (courses, labs, tutorials, peer 
tutors, direct faculty course help). 

An experiment such as this is clearly not appropriate for all 
colleges and should be entered into with great care and considerable planning. 
These ideas have been successful elsewhere, however, and may be worth trying 
at your institution. Evaluation of these ideas could be effected by using 
the first evaluation design mentioned in this section. 
Summary 

The purpose of this Special Studies section was to encourage 
colleges to make an ongoing effort at improving their programs. The first 
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three sections of thi9 document (Program Documentation, Placement Effective- 
ness, Remedial 'Program Effectiveness) focused on formative evaluation more 
as a "fine-tuning" device. In this section we hope to have encouraged you 
Xo try some more dramatic alterations occasionally. 
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ANNOTATED BIBLIOGRAPHY 



Introduction : 

The following is an annotated bibliography of a number of articles yielded 
from two different computer searches. They are listed according to: 

- Title 

- Author 

- Source 

* Annotation > 

Reports from the Education Research Information Clearinghouse (ERIC) 
System are available from any ERIC facility for a nominal fee. (You 
must have the ERIC System. No. to order) 

Thg reports are categorized according to the following areas: 

I Reports on Institutional Studies 

II Research on Basic Skills Programs 

III Basic Skills Evaluation Methodology 

IV Bibliographies, Reviews, Overviews. 

The reports in this bibliography give an overview of research done in 
the area of Basic Skills. The intent of their inclusion is to direct 
interested parties to the work done in this area. 
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I. Reports on Institutional Studies 

Title: A Follow-Ujf Study: How Do Basic Skills Students Compare 

with Non-Basic Skills Students in Some of Their Subsequent 
College Courses ? 

Author: Bergman, Irwin; Gerace, Robert 

Source: ERIC System No. ED 135435 JC 770164 

Annotation: The academic records of over 2,000 students who were required 
as a result of their national percentile scores on the 
Cooperative English Examination to take a remediation 
course in reading and writing skills during their first 
semester at Queensborough Community College (New York) 
were compared to the academic records achieved by an equal 
number of students who were not required to take a remediation 
. course. The basis of comparison used was the grades received 
by both groups of students in various introductory academic 
courses taken one semester after the remedial students 
had taken their required reading and/or writing course. 
The same survey was conducted the following year with the 
new 9lass of incoming freshmen. In both years of the study, the 
academic performance of the former remedial students with 
regard to the percentage of passing grades they received 
was very comparable to the percentage of passing grades re- 
ceived by the non-remedial students. The implications may 
be useful for evaluating programs given to underprapared 
students in two-year colleges. Extensive tabular data are 
appended . 



Title: Effects of the Basic Studies Program on the Scholastic Per- 

formance of a Selected -Group of Low-Achieving Students Enrolled 
at Bucks County Community College During the 1973-1974 
Academic Year. 



Author: Rosella, John D. 

Source: ERIC System No. ED 119794 JC 760188 

Annotation: Bucks County Community College's Department of Basic Studies 
is a comprehensive developmental education program which in- 
volves work for credit in basic academic skills — reading and 
study skills, writing and mathematics. In addition, special 
counseling is given to students in order to change negative 
habits and attitudes, and to develop a more positive self-image. 
During the 1973-74 academic year, a study was conducted to 
determine the effect of the Basic Studies Program. Students 
included in the study ranked in the bottom 40 percent of their 
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high school graduating class and scored at the 25th percentile 
or below on the Comparative Guidance and Placement Test. 
This study contrasts 86 students participating in the Basic 
Studies Program (experimental group) with 9,7 nonpar ticipants 
(control group). The average CPA earned by the experimental 
group was 2.285, while the control group earned an average GPA 
of 1.77, a difference which was statistically significant. While 
75 (87 percent) of the^&perimental group participants returned 
to Bucks for the fall 1974 semester, only 59 (61 percent) of 
the control group returned. The experimental group also pn ^d 
to be more successful in English Composition I and mathematics 
than the control group. They more frequently earned grades c 
"C M or better, and showed mote persistence. Tables of data 
are appended. 



Title: Do Remedial Programs Really Work? 

Author: Losak, John 

Source: ERIC System No. ED 045975 TM 000335 

Annotation: (Selected aspects of the remedial reading-writing program of 

Miami-Dade Junior College were evaluated. Placement in the , 
program was designed as the independent variable. Grade point 
average, reading and writing test scores, continuation in 
college, and performance ii» regular college courses were de- 
pendent variables. Student c earning a raw score of 22 or less 
on the School and College Ability Test, Form 1A, Verbal, were 
classified as academically underprepared for college-level work 
and were required to enroll in the remedial program. Results 
indicate that the program does not produce any meaningful dif- 
ferences in student withdrawal from college, is not effective 
in raising grade point average during the second semester of 
college emcllment to a "C" level, and does not result in 
significantly higher scores on a reading or writing test 

i when compared with the control group scores. The remedial 

program produced no differential effects by race or sex. 
A paradigm for devising remedial programs is suggested. 

Title: A Multivariate Analysis of a Special Studies Program . 

Author: Brown, Sidney E. ; Ervin, Leroy 

Source: College Student Journal, vl2 n4 p379-81 Win 1978 78 

Annotation: The differences between two groups of students (17 Blacks 

and 23 Whites) were examined on the basis of participation and 
nonparticipation in a special studies program designed 
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primarily to meet the needs of minority and disadvantaged 
students. Participating students met with more academic 
success than nonparticipating, irrespective of race. 



Title: A Study of Remedial Reading Courses (BE-03) Offered During 

the Fall, 1972 Semester: A Baseline for Longitudinal Studies. 

Author: Cohen, Edward G. : And Others 

Source: ERIC System No. ED 144653 JC 770466 

Annotation: This study was designed to determine whether or not students 

assigned to remedial reading courses at Queensborough Community 
College were actually improving their reading skills. Students 
(1,543) who scored below 33Z on the Reading Comprehension 
Subtest of the Cooperative English Test (C0(5P) were divided 
into four classes of ascending ability and two mixed classes. 
At the end of the semester all were post-tested using the 
same test. Scores were compared in vocabulary, level of 
comprehension, speed, and total reading. Mean scores for 
each class increased significantly in all four variables. 
Analysis of variance tests showed the groups to be distinctive 
from each other, and significantly different on each variable. 
Students at each reading level progressed on the post-test to 
the level of the pre-test of the next highest group. A com- 
parison of final grades and the COOP post-test showed more 
agreement in classes of higher reading ability; 32% of those 
in the highest group failed COOP, and .6% of that group 
failed the course, while 77.6% of the lowest group failed COOP, 
and 33.3% of that group failed the course. Overall, 40.2% 
reached an acceptable reading level accenting to COOP, but 
final grades yielded a success rate of 86.9%. Tukey multiple 
comparison tests on pre- and post-test scores and frequency 
tables for the four reading levels are appended. 



Title: 



Author : 



Description of a Compensatory College Education Program For the 
Disadvantaged and Its Associated Research and Evaluation Program . 

,Spuck, Dennis W. and Others. 



Source: ERIC System No. ED 042827 

Annotation: This paper reports on a large-scale project of research and 
evaluation of a program for disadvantaged minority group 
students conducted by the Center for Educational Opportunity at 
the Claremont Colleges. The Program of Special Directed Studies 
for Transition to College (PSDS) , a five-year experimental project 
is aimed at providing a four-y^ar, high quality college education 
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for educationally and economically disadvantaged minority 
group students. Forty students were admitted in 1968 with 
full financial support. A three-week orientation program 
tailored their course load to their abilities and totarests, with 
adequate and individualized counseling. Two years are allowed 
to qualify for regular admission with no grades recorded on the 
permanent transcript during this time. The research intrinsic 
to the program is for the purposes of: (1) investigating 
the consequences of special collegiate arrangements; (2) 
evaluating and accounting for areas of success and failure; 

(3) making research available locally and nationally; and, 

(4) providing bases for desirable changes in PSDA and other 
programs. The research design is a pre-test, post-test 
control group allowing for analysis of test scores. The 
multivariate definition of "success" and evaluative measures 
to be used are included. 



Title : The University Learning Laboratory: Meeting Student, Needs 

in the '70's . 

Author: Nayman-Robbie. and Others 

Source: ERIC System No. ED 128688 

Annotation: College and University Learning Centers have evolved from narrowly 
defined remedial services to agencies serving the entire 
student community. ' The changing role of Learning Centers 
necessitates systematic program evaluation and assessment of 
student needs. Dining 1975; the Colorado State University Learning 
Laboratory staff collected data on client characteristics and 
client perceptions of learning laboratory programs. Results 
are presented and major findings are discussed. (1) The 
University Learning Laboratory serves a diversified clientele 
including students with all levels of academic functioning. 
(2) Needs of learning laboratory clients can be appropriately 
categorized as remedial, preventative and developmental, fol- 
lowing the model of Morrill, Oetting and Hurst (1974). (3) 
A large majority (70-80%) of clients responding to the survey 
had favorable attitudes toward learning laboratory services. 
(4) One-half to two-thizds of clients responding to the survey 
agreed that the learning laboratory had helped them to develop 
skills, improve academic performance and enrich their learning 
experiences. (5) Recommendations are presented for continuous 
assessment of student needs to allow the learning laboratory 
to respond to changing campus conditions. 
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Title: Evaluating and Auditing a Community College Learning Skills 

Center Program: Second Year . 

Author: Cohen- Benjamin, Barbara: Olson, Gerald T. 

Source: ERIC System No. ED 138331 JC 770263 

Annotation: An evaluation of Los Angeles City College's Learning Skills 
Center (LSC) was conducted in order to ascertain the nature 
of the LSC f s outcomes. The LSC offers students an opportunity 
. to participate in diagnositic and prescriptive open-entry 
open-exit programs in an individualized setting. Types of 
programs available through the LSC include communication 
skills, quantitative skills, and tutoring in other college- 
level courses. The evaluation of the LSC considered three 
specific areas: the first consisted of analyses of data 
gathered in relation to specific program objectives, the 
second examined unexpected spinoffs, and the third con- 
sidered the humanistic characteristics of the LSC. Results of 
the evaluation indicated: (1) students who utilized the LSC's 
services in the areas of basic math, basic English, accounting, 
business, and chemistry were associated with significantly 
lower dropout rates than were students who enrolled in these 
courses but did not utilize the LSC; (2) the LSC had gained 
the support of the Academic Senate and a number of departments 
traditionally opposed to remedial courses; and (3) numerous 
students who had /used the LSC offered favorable personal evaluations 
of the Center. A critique of the causal-comparative method em- 
ployed in the study is included. 



T it le : Factors in Remedial Education. The Case Study of an 

Alternative School . 

Author: Millonzi, Joel; Kolker, Aliza 

Source: ERIC System No. ED 135425 JC 770154 

Annotation: The Middle College High School, an experimental school at 

LaGuardia Community College (New York) which incorporates the 
last three years of high school and the first two years oi 
college, was designed for adolescents of average ability 
but deficient computational and communication skills. Re- 
mediation was viewed as a social as well as instructional 
process;. the remedial strategy involved academic and personal 
counseling and basic skills instruction in a small, personal 
setting. This study was designed to isolate and analyze 
the remedial aspects of the program s social structure and 
functions. Participant observation data were collected over 
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a three-semester period, focusing on instructional methods and 
content, .motivating devices and failure management techniques, 
counseling patterns, discipline management, and interface 
between the Middle College and the college. Interviews were 
also conducted with staff and samples of students. The data 
indicated that the strongest component of the program was its 
small size and that the college environment facilitated student 
motivation,* but that the counseling component fell somewhat 
short of its potential due to teacher/counselor uncertainty 
over non-traditional role expectations. The overall impact 
of basic skills instruction was found to rank behind that of 
size, setting, and counseling. 

Title: A Study of Remedial Algebra Courses Taught At Queensborough 

Community College: Spring and Fall 1973. 

Author: Cohen, Edward G.; Diamond, Arnold H. 

Source: ERIC System No. ED 144654 JC 773467 

Annotation: A study of 159 students assigned to remedial algebra courses 
was initiated at Queensborough Community College to determine 
levels of improvement in algebrax. skills and the effects of 
regular, programmed, and modular teaching methods . All students 
studied the same material with the following differences: (1) 
regular instruction used no programmed or modular techniques; 
(2) programmed sections used pre-testing to decide individual 
areas for study; and (3) module students were placed in one of 
four instructional levels according to pre-test scores, and com- 
pleted £*om one to four modules. The Cooperative Math Test: 
Algebra 1, was used for placement/pre-test , and as a post-test 
after final class examinations. Mean scores of all sections 
increased from pre- to post-test; all increases were statistically 
significant. Analysis of covariance showed no statistical 
difference among means of the three instructional groups. 
However, modular .and programmed methods required less student 
time for completion, although only in modular sections could 
students actually complete the course in three, six, or nine 
weeks. Sampling was dependent on teacher cooperation. Future 
research relating to all remedial sections is recommended. Mathe- 
matics requirements by curriculum are appended. 



Title: Evaluation of College Learning Center Instruction in 

Six Subject Areas, Spring 1975 . 

Author: Tatham, Elaine L. ; And Others 
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Source: ERIC System No. ED 110141 JC 750448 

Annotation; This study evaluates the success of Johnson County Community 
College's College Learning Center (CLC) In helping students 
to Improve specific skills In six areas: spelling, vocabu- 
lary , English, mathematics, reading comprehension, and reading 
rate. The CLC offers individualized programs which employ a 
combination of self-instructional materials and individual 
tutoring. Data were analyzed separately for the spring 1972- 
summer 1973 and fall 1973-summer 1974 periods in order to detect 
any difference in the success patterns of CLC instruction. 
Evaluation was based on analysis of gains in performance for every 
student participating during either period. The average gains 
represented significant improvement from pre-test to post-test 
in all six areas; the improvement was similar for both time periods. 
Statistical data is presented graphically for each subject. 
Appendix A describes the six courses and their pre- and post- 
tests. Appendix B lists supplementary instructional materials 
for the six programs. 



II. R esearch on Basic Skills Programs 

Title: Proceedings of the National Conference on Remedial/ 

Developmental Studies in Post-Secondary Institutions. y 77 

Source: ERIC System No. ED 152882 

Annotation: This Conference was called to discuss the need for, and the 
strategies used in, Implementing Basic Skills Remediation 
at the College and University Level. All levels of Higher Edu- 
cation were represented, as well as research professors, elementary, 
secondary, and post-secondary .school administrators and 
teachers, representatives of the business and governmental 
communities and individuals from the community at large. 
Among the issues that were discussed were whether universities 
should be teaching remedial courses and whether credits should 
be given- for these courses when they are taught. It was noted 
that when thinking about programs of remediation, writing should 
be stressed instead of reading. The assumptions that guide 
remediation programs, and the goals of remediation programs must 
be carefully thought about prior and during program planning. 
It was noted that although universities have made an effort to 
seek minority students this trend has declined. In addition, 
although universities can be elitist with regard to their 
admission policies there are some that have departed radically 
from this position. Both of these factors shape the form of 
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programs that intend to remediate the difficulties experienced 
by minority students. Furthermore, whether programs should be 
based on the disadvantagement pf students, or the differences 
among students is another important concern guiding remediation/ 
developmental programs. 



Title: Developmental Program Primes High-Risk Students . 

Author: Milander, Henry M. ; Simmons, George A. 
Source: ERIC Systems No. EJ 040939 

Annotation: As community colleges open their doors to a wider range of 
students a variety of programs must be designed to develop 
basic skills in underachievers. 



Title: Basic Skills Programs: Are They Working? 

Autho_: Grant, Mary Kathryn; Hoeber, Daniel R. 

Source: ERIC System No. ED 159918 HE 009659 

Annotation: Basic Skills programs are now almost a given in the curricula 
of postsecondary institutions; however, the key area of 
concern oi basic skills researchers is, Are they working? 
The answer is two fold: yes, those involved in and committed 
to the programs are working very hard, and no, the programs 
themselves .ire not working very well. There is a dearth of 
empirically based evaluation research on the programs. The 
following are important in the ^discussion of basic skills 
programs: a historical perspective of governmental and 
institutional policies that have been challenged by the need 
fot basic skills programs: the nature of the student who 
enters a program; a generalized description and definition 
of basic skills programs; and the conclusion drawn by some 
researchers that the existing compensatory programs have 
done very little toward eradicating the academic problems 
of disadvantaged college students. 



Title: A Cetyter for Communications Skills. 

Author: Coons, Daniel E. 

Source: E&IC System No. ED 127607 

Annotation: The nationwide problem of declining communication skills is evident 
in the large numbers of students entering college with de- 
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ficiencies in reading, writing, and speaking skills. This paper 
discusses the operation of a communication skills program within 
a college communication skills center which functions as a sup- 
portive resource service and provides students with the basic 
communication tools for academic achievement and self-image 
improvement. The discussion focuses on reading, writing, 
and speech programs, »s well as on the staffing of a center, 
on the selection of students for a skills program, and on the 
evaluation of a skills program. The paper concludes that the 
success of a communication skills center may be measured by the 
number of students completing the program, by the reduction 
in the college drop-out rate, and by the improvement in test 
scores n graduate students. 

Title: Opportunity Programs for the Disadvantaged in Higher 

Education. 

Author: Gordon, Edmund W. 

Source: ERIC System No. ED 114028 

Annotation: Numerous federal, state, and institutional projects have been 
undertaken in the areas of current educational opportunity 
programs for the disadvantaged, but there is still a need to 
bring about significant changes in the areas of administration, 
curriculum, program evaluation, and financial assistance. 
The state of compensatory education at the college level and the 
equivocal status of evaluation efforts is confused, but some 
conclusions can still be drawn. Where programs have been imple- 
mented with full systems of student support services, special 
opportunity students showed equal or higher grade-point 
averages than regular students of compa -able ability, equal 
or higher retention rates, and increased self-esteem and 
motivation. Where special opportunity students are selected 
on the basis of previously demonstrated talent, college com- 
pletion rates exceed those of the traditional college population. 
Some student support services that show promise include: full 
systems of student support services, services that provide pro- 
tection from an impersonal atmosphere, remedial courses based 
on specific needs, programs which give attention to the socio- 
political life or the students, behavior modification, course 
content that complements the nationalistic concerns of 
students, developmental programs in test taking and study habitb, 
and programs that prepare adolescents In the tranp'tion 
from high school to college. 
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Title: Developmental Education: 

and Evaluation. 



A Primer For Program Development 



Author: Roueche, John E. ; Roueche, Suanne D. 
Source: ERIC System No. ED 148237 

Annotation: Addressed in this book is the problem encountered by insti- 
tutions as more students have entered colleges without 
adequate preparation in the basic subject areas — communications, 
mathematics, and science. To accommodate the diversity in 
the new student populations, varying instructional methods 
are required. Developmental education programs need therefore 
to be measured for quality and effectiveness. Aspects 
of program development to be considered are: an overview 
of developmental studies programs; institutional c fitment; 
program design components; staffing; and consolidating 
all the elements. Program evaluation takes into consideration 
the essential data and personnel, andthe development of an 
evaluation design for development al/i>aticat ion. A survey 
instrument for program evaluation is included, along with 
a four-page bibliography. 



Title: 

Author: 
Source : 



Equal Opportunity in Higher Education. The Current State 
of the Art and Recommendations for Change and a Review 
of the Studies of the Programs. 

Gordon, Edmund W.; Fahrer, Kimberly. 

ERIC System No. ED 123331 



Annotation: These two articles focus on opportunity programs in Higher 
Education, the first of which notes that the state of the 
art of designing and delivering these programs is confused, 
contradictory, and yet encouraging. Positive and negative 
research findings and program variation contributes to the 
confused state of the art. Another problem concerns the 
indicators used to define the target population. These may 
be insufficiently sensitive to either plan adequately for 
educational experiences or to evaluate program effectiveness. 
The status of the programs is contradictory in part because 
the roles and purposes of the institutions in which they exist 
are contradictory. Overall, effects of programs is positi/e, 
even if limitedly so. When adequately supported and implemented, 
college completion' rate is high and attrition low. \mong 
recoumendations for change include the following: the establish- 
ment of free or low cost continuing post secondary education, 
development of improved diagnostic assessment technology, and 
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matching of student characteristics to institutional capabilities. 
A review of the studies of the program is provided in the 
second article, which notes that policy changes have occurred 
in three broad areas In the structure of Higher Education: 
Admissions/recruitment , financial aid, and curriculum. The 
practical responses to the necessity for change in these areas 
are then discussed as well as the theoretical issues Involved. 



III. Basic Skills Evaluation Methodology 



Title: Needed: Systematic Evaluation of College Remedial 

Reading Programs . 

Author: Deem, James M. 



Source: ERIC System No. ED 163410 

Annotation: Although, as an important component of any basic skills 

curriculum, college remedial reading programs need to be objectively 
and fully evaluated, no comprehensive, systematic procedures 
have been developed for this purpose. Three guidelines should 
be followed in evaluating college remedial reading programs: 
first, appropriate variables that allow for maximum input must 
be selected; second, the evaluation must cover not only the 
performance of the students in the program but the entire 
program itself; and, finally, the end result of the evaluation 
must be program improvement. Since no complete evaluation 
model has been developed to aid college remedial reading 
program directors, directors must rely on the few evaluation 
models that have been proposed. Until meaningful systematic 
evaluations of college remedial reading programs can be carried 
out, the success of such programs and their various methodologies — 
in short, the need for such programs — will be seriously questioned. 
(The paper Includes an outline of one model for program 
evaluation and a bibliography of relevant readings.). 

Title: Human Diversity, Program Evaluation and Pupil Assessment; 

And, A Critical Review of frlack Consciousness, Identity 
And Achievement . 

Author: Gordon, Edmund W. ; Grannis, Joseph C. 

Source: ERIC System No. ED 141451 

Annotation: The first of two papers included in this document addresses 
two related prob 1 ems: problem one: the appropriateness of 
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existing standardized tests p{ achievement for the assessment 
of academic function in minority and disadvantaged group member 
students. Problem two: the appropriateness of such instru me nts 
for the assessment of the impact of large-scale educational 
programs. Gordon asserts that "The problem of large-scale 
evaluation... is larger than one of what kind of achievement 
tests to use." It may be that we could endure the problems 
related to the tests if we were better able to deal with such 
problems as the following: 1) The nebulousness and variabilis 
of treatments, 2) The complex economic, political, and social 
context in which the treatments are set, 3) The diversity 
of populations served and goals sought, 4) The reconciliation 
of necessary and sufficient conditions, and, 5) Such limitations 
of evaluative research technology as: program and population 
specification, program and population sampling, interchangeable 
and dialectical nature of the dependent and independent variables, 
inappropriateness of extant statistical analyses for the study 
of the dynamic blending of variables by which effects may be 
explained, the policy of the best generic treatment, and, normative 
approaches to aggregate data in search of relationships that 
may be idiosyncratically expressed. The seco.;J paper in this 
document critically reviews the book "Black Consciousness, Identity 
and Achievement," by Patricia Gurin and Edgar Epps, New York: 
John Wiley, 1975, 



Title: 



Higher Education Equal Opportunity Program: 1974-75 Annual 
Report. Pennsylvania Dept. of Education, Harrisburg. 



Author: ftrehman, George E., Jr.; McGowan, Kristine A. 



Source; 



Annotation: 



\ 

EilC System No. Ed 154047 

Par \_° f thiS report gives the 1974-75 survey findings and 
distribution of Quality Point Average (QPA) of the 44 programs 
set up under Pennsylvania ' a Higher Education Equal Opportunity 
Program (Act 101) established in. 1971. The program is supported 
entirely by State funds whic* are distributed to postsecondary 
degree granting institutions to maintain learning and special 
counseling services for disadvantaged undergraduate students. 
The report also provides an analysis of the relative effectiveness 
of the tutoring and counseling aspects of the Act 101 programs 
in Pennsylvania's institutions of Higher Education. It covers 
specifically the design of the study and findings. The findings 
are discussed separately as fellows: (1) tutoring effectiveness 
by institutional category; (2) counseling effectiveness by 
institutional category; (3) institutional ranking for tutoiing; 
(4) institutional ranking for counseling; and (5) patterns 
of overall program effectiveness. It is concluded that there 
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Title: 



Author: 



is a positive correlation between hours of intervention 
(either tutoring or counseling) and achievement 
(Quality Point Average). 



SLatevide Assessment of Developmental/Remedial Education 
at Maryland Community Colleges , 

Linthicum, Dorothy S. 



Source: ERIC System No. ED 175514 JC 790505 

Annotation: A study was conducted to determine the scope and characteristics 
of developmental/remedial activities in Maryland's 17 community • 
colleges in terms of objectives and goals, courses and activities, 
student information, costs and revenues, and administration 
and organization. The study revealed: (1) 15 colleges offered 
developmental education for credit in English and mathematics; 

(2) 15 colleges offered courses in reading and/or study skills, 
and provided tutoring, counseling, and self-paced instruction; 

(3) total enrollment in remedial courses in fiscal year (FY) 
1978 was more than 42,000—30,000 in credit courses and 12,000 
in continuing education courses; (4) direct costs for FY 1978 
totaled about $3.4 million (direct remedial instructional cost 
for each remedial full-time equivalent (FTE) student statewide 
was $931); and (5) developmental programs tended to be de- 
centralized, with most programs conducted within academic 
departments* Another aspect of the study involved measuring 

a sample of students enrolled in developmental English courses 
at eight colleges against the success of a sample of students 
enrolled in English 101. Developmental students tended to 
complete fewer college-level English courses and make lower 
grades than the control group. The study report considers the 
implications of the findings in terms of placement, open 
admissions, professional development, organizational structure, 
evaluation, and funding. 



Title: Evaluating the Eff e ctiveness of the Special Higher Education 

Opportunity Programs Financially Assisted by New York State . 

Source: ERIC System No. ED 105054 

(Expenditure Review, NY State Legislative Commission) 

Annotation: This program Audit. evaluates the effectiveness of the Special 
Higher Education Opportunity Programs financially assisted by 
New York State in the four year public and private Univer- 
sities and Colleges in the State. These programs are officially 
identified as: seek — search for education, elevation and 
knowledge (1966) — the program at the City University of New 
York; EOP— Education Opportunity Program (1967)— the program 
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at the State University of New York; and HEOP—Higher Education 
Opportunity Program (1969)— the program at the private Universi 
ties in New York State. This audit compiles statistics to 
evaluate and compare the performance of the programs for the 
disadvantaged in selected units of participating Universities 
and Colleges. Comparisons are made between the "disadvantaged" 
group and the "regular" College population and between the 
"disadvantaged" group and students in the "open" admissions 
group at the City Universities. Information and statistics 
used in the audit come from three primary sources: reports 
which have been submitted by the State Education Department, 
SUNY and CUNY, a detailed sample questionnaire asking for 
specific information in the same format for 26 selected schools 
and a 10 percent random sample of the average annual enrollment 
in the program for the disadvantaged at these selected schools. 



Title: Evaluation of Remedial Programs in Community Junior 

Colleges: Community College Presidents 1 View . 

Author: Gwynne, Margaret; And Others 

Source: ERIC System No. ED 089810 JC 740119 

Annotation: This report describes a survey which was conducted to determine 
community junior college presidents' views of how to evaluate 
remedial programs. Questionnaires were mailed to 166 schools 
in the New England and Middle Atlantic States. Since no insti- 
tution forwarded a copy of any formal evaluation guidelines, 
it is suggested that very few of the community college sample 
have any formal evaluation of remedial education courses or 
programs. It was found that the faculty was involved in 
program evaluation in more than half the community colleges 
responding and the academic dean in slightly less than half. 
The questions and responses are included in the report. 

Title: An Analysis of a Multidimensional Success: Measure for 

PSDS Students. 

Author: Spuck, Dennis W. 

Source: ERIC Systems No. ED 043667 TM 000115 

Annotation: The Program of Special Directed Studies (PSDS) identifies 

persons with intellectual ability whose academic achievement, 
as indicated by traditional measures, is inadequate to secure 
admission to selective colleges. Then, through a program of 
supervised college courses, special services and individual 
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tutoring, it attempts to prepare these students for a standard 
degree program. This study seeks to expand the concept of 
success, traditionally measured by grade point average, into 
a multidimensional measure which includes grade point average, 
student self -evaluation, and evaluation of students by faculty. 
These evaluations are compared with those of regularly en* 
rolled students. A detailed description and analysis of the 
data derived from the 40 students involved in the program at 
the Claremont Colleges is presented. Use of this multidi- 
mensional measure of success seems to indicate the effectiveness 
of PSDS. 



Title: A Systems Analysis and Evaluation of a Junior College 

Developmental Studies Program . 

Author: Sparks, June R. ; Davis, Cynthia L. 

Source: ERIC System No. ED 13 892 JC 770231 



Annotation: This paper describes the structure and operation of an 

evaluation system for the developmental education program 
at Dalton Junior College (Georgia). All applicants for 
admission to the college who score below 330 on either section 
of the Scholastic Aptitude Test are required to take the 
Comparative Guidance and Placement (CGP) test to determine 
their proficiency in various skills. Those falling below 
the institutional cut-off point on the CGP are required 
to complete Special Studies courses before being admitted 
to college credit courses., Students may exit from a Special 
Studies course by completing the course requirements and 
scoring above the cut-off point on the CGP in that area; 
otherwise they are advised to enroll in the next sequential 
developmental course. In order to monitor student progress, 
an evaluation system is utilized which allows identification, 
testing, registration, and accounting for every Special 
Studies student. This system also allows followup 
studies of subsequent student progress in credit courses. 
Among the methods used to evaluate the program are analyses 
of CGP score differentials and retention rates in subsequent 
credit courses. Included in the paper are graphic repre- 
sentations of the Special Studies program and the 
evaluation system. Special Studies course descriptions 
and content objectives are appended. 
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IV. Bibliographies, Reviews, Overviews. 



Title: 

Author: 
Source : 
Annotation: 



Special Secondary School Programs For The Disadvantaged : 
An Annotated Bibliography of Doctoral Dissertations . 

Jablonsky, Adelaide Comp 

ERIC System No. ED 102223. 

This annotated bibliography is the seventh in the doctoral 
research series. It encompasses doctoral research on "Special 
Secondary School Programs for the Disadvantaged 11 reported 
in "Dissertation Abstracts International 11 from 1965 through 
June 1973. The citations are arranged in the following 
categories: College Preparatory Programs (Upward Bound, 
Project Opportunity, Other Programs), special programs 
(Vocational, Outward Bound, Other Programs), follow-up 
studies, and history and evaluation. Under these categories, 
citations are presented in order of year of completion. Where 
a number of citations appear in a category and in the same 
year, they are arranged in alphabetical order by name of 
author. A subject index, an author index, and an institu- 
tion index are appended. 



Title: 
Author : 
Source : 
Annotation: 



Developmental Education: A Rapidly Expanding Function , 
Lombard!, John 

Community College Review, v7 nl p65-71 Sum 1979 79 

Discusses four phases of developmental education in community 
colleges: pretransfer, handicapped, remedial, and adult 
basic education (ABE). Reviews literature regarding 
enrollments, program costs, faculty attitudes, and program 
effectiveness in each area. 



Title: 
Author: 
Source : 
Annotation; 



Developmental Education: Does It Work?: An ERIC Rev J ew . 
Hill, Andrew 

Community College Review, v6 n2 p41-47 Fall 1978 78 

Reviews developmental education programs stressing the need 
for institutional research to determine the viable paits of 
such curricula in view of cutbacks being made of costly 
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programs. Discusses factors causing insufficient skills in 
students, considers program evaluation, and reviews six 
established developmental programs. 



Xitle: Overview of Developmental Studies for Occupational Students^ 

A Sourcebook for Post-Secondary Programs. Research 
Publication 76-4. 



Source: ERIC System No. ED 128031 JC 760467 

Annotation: This publication is intended to be used as a sourcebook 

and reference by practitioners interested in postsecondary 
developmental programs, with a particular focus on occupa- 
tional students. The information presented was derived from 
a statewide survey of staff members involved in the operation 
of developmental programs in 51 two-year colleges and 10 
Educational Opportunity Centers in New York State (58% 
responded). Part I, State of the Art, compiles the survey 
data regarding college and student characteristics, develop- 
mental program characteristics, program components, and 
ratings of program success. Part II, Program Descriptors, 
describes a composite mythical most successful developmental 
program, and compares survey responses by the various types 
of institutions surveyed to the composite profile. Part Hit 
Program Resources, lists recommended resources for reading, 
writing, study skills, math, and vocational-personal/decision- 
making programs in regard to desired outcomes, standards of 
success, strategies/approaches, measurement tools, instructional 
materials, and modes of instruction, and compiles consensus 
reco mme ndations for programmatic standards. Part IV, Human 
Resources, lists respondents willing to serve as contact 
persons. 



Title: The Junior College Remedial Program . 

Author: Roueche, John E. 

Source: ERIC System No. ED 013661 JC 670938 

Annotation: A review of* 20 documents in the ERIC Clear in^hou** tor 

Junior College information was the basis for this report on 
remedial instruction* Most Junior Colleges, having an ' Jpen 
Door" admission policy, are enrolling increasing numbers of 
low ability students, and accept remediation as a legitimate 
function. While remedial programs are common among Junior 
Colleges, mo**: available material about such programs is 
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descriptive, and there is little research evidence of their 
effectiveness. Lack of success appears to be due to failure 
to formulate appropriate objectives, inadequate student selection 
processes, and lack of teachers qualified for such programs. 
Research is needed for evaluation of present programs and for 
a foundation on which to build new remedial services, (WO) 
Descriptors: *Junicr Colleges/ *Low Ability Students/ 
*Low Achievers/ Program Evaluation/ *Remedial Programs/ Re- 
search. 



Title: The Effectiveness of Secondary and Higher Education 

Intervention Programs: A Critical Review of Research . 

Author: Sherman, Roger H. ; Tinto, Vincent 

Source: ERIC System No. ED 106378 

Annotation: This paper reviews and synthesizes the available literature 

concerning the effects of intervention programs at the secondary 
higher educational levels. In achieving an understanding 
of the design of these efforts it is important to recognize 
that the first projects, e.g., the Demonstration Guidance Project, 
Higher Horizons, established a virtually universally followed 
model of educational intervention which: (1) employed a 
"deficit" mo^el to account for differential rates of academic 
achievement; (2) offered supportive educational services; 
(3) worked with a segment of the "disadvantaged" population; 
and, (4) concentrated on the development of reading and 
mathematical skills. While most of the studies considered 
possess serious methodological weaknesses, the resulting 
evidence nevertheless suggests that the projects have in- 
creased the numbers of students graduating from high school 
and applying to, enrolling in, and graduating from college. 
The findings also indicate some positive impact in the areas 
of academic values, attitudes and motivations. But neither 
the gap in academic achievement between "disadvantaged" and 
''advantaged" students nor the academic achievements of partici- 
pating students relative to non-program students from 
similar backgrounds seems to have been affected. 
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HISTORICAL PERSPECTIVE - College B 

The placement and remediation progams at College B predates the 1978 
State Board of Higher Education mandate that all entering freshmen at New 
Jersey state and community colleges be evaluated for basic skill deficien- 
cies in reading, writing, and mathematics. 

In 1972 the college decided to adopt a "learning center" concept as a 
foundation for the development of a full scale tutorial program. This early 
attempt to assess the extent of underpreparation of entering freshmen re- 
sulted in the establishment of two elective reading courses. In the spring 
of 1974, as the placement and remedial programs continued to evolve, the 
Faculty Senate formally adopted a resolution expressing their concern with 
the underpreparation of entering freshmen establishing a Basic Skills Coun- 
cil charged with documenting the learning center idea. The Council was 
commissioned to study the questions of needs assessment, placement, and 
remediation. This task involved a comprehensive review of the literature 
on basic skills and an examination of how and what other colleges were do- 
ing to meet this growing problem. 

From this experience grew a project initiated in September of 1975. 
The experiment involved testing all entering students in an effort to 
assess the extent of their underpreparation. From this pool of subjects, 
two matched samples of students, identified as being underprepared in two 
or three areas (reading, writing, mathematics), were selected. The con- 
trol group (80 students) was tracked into the regular freshman curricula, 
while the experimental group (80 students) was identified as underprepared 
and required to take courses to remediate their skill deficiencies. Built 
into the structure of this experimental program was an evaluation proce- 
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dure that established three criteria for determining the success of the 
progf am: 

(1) student grade point average—predicted vs actual; 

(2) an analysis of pretest/posttest gain scores; 

(3) an analysis of attrition/retention rates. 

The results of the study suggested the potential benefits to be de- 
rived from a systematic basic skill assessment process, and placnnent and 
remediation programs by revealing a statistically significant differences 
in all three categories between the experimental group and the control 
group. 

The favorable outcomes cited abpve led to the decision by the college 
administration to continue the program into the 1976-77 academic year, and 
to follow and evaluate the progress of the 1975 group. The Council follow- 
ed the same procedure adopted in 1975 by selecting two matched samples. An 
evaluation of the program's effectiveness again suggested that an establish- 
ed procedure fu. identifying and remediatinq basic skill deficiencies could 
nake a favorable difference in: (a) student grade point average, (b) pre- 
test/posttest gain scores, and (c) attrition and retention rates. The suc- 
cess of the two-year project led to the Council's recommendation and the 
college administration's approval to, beginning in the fall of 1977: 

(1) test all entering students; 

(2) identify those students who are underprepared or deficient in one 
or more basic skill areas (reading, writing, mathematics); 

(3) place those identified students in a required, structured program 
of remediation; and 

(4) require all freshmen to take a one-credit "survival skills" course 
(Freshman Seminar). 
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The College B experience has led to the establishment of a fully-staffed 
Academic Development Office, which assumes responsibility for; identifying 
skill deficiencies, placing students in the appropriate courses, and remedi- 
ating skill deficiencies. In addition the Academic Development Office pro- 
vides all students with labs/tutorials in the basic skills areas as well as 
counseling, program evaluation, staff recruiting, and inservice training. 
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BURLINGTON COUNTY COLLEGE 
Pemberton-Browns Mills Road 
Pemberton, NJ 08068 609-894-9311 

BASIC STUDIES PROGRAM 

Chairperson: Ms. Joy R. Hughes 
January 1980 

The Basic Studies Program for full-time, first-time freshmen aims to 
devrV.p the student's ability to succeed academically at the college level 
The program i^ developmental rather than remedial in nature since its curr 
iculum and its goals are concerned with the development of the student as 
an eager independent learner, and because it aims to provide the student 
with a whole range of cognitive, affective, social and political exper- 
iences that bridge the gap' between high school a„j college. 

The majority of the students in the Basic Studies Program are ; 2cent 
high school gradautes wi uh average^or above average academic records from 
schools or programs .within schools that did not provide, and in some cases 
did not intend to provide students with the comprehensive and intense col- 
lege preparatory work that is needed for success in college. Most of our 
students, £or example, did not have four years of col lege* preparatory math 
ematics or four years of college preparatory English with rigorous writing 
reoji rements. As a result, the students, though in every sense entitled 
to have graduated from high school, are not adequately prepared to succeed 
in college. A minority af the BSP students either did not do well in high 
school or did not complete high school and, as a result, these students 
may require more time to complete the program. 

The majority of the BSP Students complete the program in one semester 
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anc then, with the assistance of their BSP counselor, select a balanced, 
full-time program of typical first-year college courses in the second semes- 
ter. Some of the students proceed at a fa£ter\rate and, therefore, receive \ 

credit for a higher level course(s) in th^ir first semester; others need to 

v j 

spend more than one semester in the Basic Studies Program. In all cases 
grades and credit foi the BSP courses or moTe^advanced courses are based on 
published criteria which include competency-bas^d performance measures. 

Approximately sixty-five (65) -percent of the first-time feshmen in 
Fall 1378 had test scores that indicated they should be enr Med in the 
Basic Studies Program. Due to funding limitations and other constraints, 
however, only full-time students planning to attend day classes on the 
Pemberton Campus were placed in the program. The remainder of the students 
eligible for the program were advised by counselors in the .Student Develop- 
ment Division to enroll in one or more of the pre-college level courses 
provided by the college to prepare students for college level work. In 
order to carry out the charge of the N.J. Chancellor uf Higher Education 
to require students with pre-college level courses, arrangements have be^n 
made to offer a day and an evening Basic Studies Program at the new Cinna- 
minson Center in Winter 1979, and it is expected that an evening program 
on the main campus in Pemberton will be offered by Fall 1980, *o that stu- 
dents will be able to take courses in locations relatively near their ^omes 
no matter in which section of the County they reside, 

The overriding purpose of the BSP and, indeed, its raison d'etre is to 
provide our students with the skills, experiences, and knowledge needed to 
succeed in their college program so that the extremely high attrition rate 
of BCC's first-year, ,ull-time students can be legitimately reduced to an 
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acceptable level. The staff of the Basic Studies Division agree that it is 
regrettable tht students who have satisfactorily completed the requirements 
of their respective high schools should not be able to enter immediately into 
regular college level programs. Many programs in high school, however, 
are not intended to ,a college preparatory in scope and depth; therefore, 
many of our freshmen are not adequately prepared for college and will need 
to spend a semester or, in some cases, two semesters in the Basic Studies 
Program. 

So that students may complete their college preparatory work as quickly 
as possible, the curriculum and out-of-class assignments have been coor- 
dinated for maximum effect by the faculty 0 n the team that services a par- 
ticular set of students. (A team consists of 1 Social Science Instructor, 
1 Writing Instructor, 1 Reading/Study Ski Ms Instructor, 1 Counselor, 1-2 
Head Tutors, peer tutors, and students). This coordination also reduces 
the time our students, many cf whom work full or part-time, need to spend 
on out-of-class work and s-.ould reduce the amount of time an individual 
faculty member needs to spend on routine correction of assignments so that 
faculty can spendd more time with students. 

The statistics listed below refer to the students enrolled in the Basic 
Studies n rograrc in Fall, 1979, the first semester of the program. 
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CAMDEN COUNTY COLLEGE 
WRITING PLACEMENT GUIDELINES 
FOR THE NJCBSPT 



Comp 1 
R 

BWS2 
BV:S1 

X 



Composition 
Score : 



165 - CM 



Comp 1 



(63 


- 64 










Essay 


6-8 


4-5 


2-3 


Comp 1 


R 


X 



Engli h Composition I (3 credits) 

Referral. Students may enroll in English 
Composition I, but must report to the 
Writing Lab for tutorial assistance. 

Basic Writing Skills II. Second level 
developmental course. (3 non-degree credits) 



Basic Writing Skills I 
dcvel oprr.cn t a 1 course . 
(3 non-degrce credits) 



First level 



(35 - 62 1 



Sentence Structure 



57-81 



53-55 35-52 



Essav 


6-8 


2-5 


X 


BWS1 



Sentence Structure 
Subscores 162 


7 or above 
on one or 
both 


6 or belov; 
on both 





6-8 


2-5 


%# 
A 


3l?S 1 



Certain scores are contradictory. 
Essays should be read analytically and 
reading placement checked. 



Lopi ca J Kcl ationships 
Subscore 2&4 


8 or above 
on one or 
both 


7 or belov; 
on both 


r BUS 2 



Essav 


7-8 


6 


Li»-5j 


2-3 


Comp 1 


R 


BUS2 


X 



Note: When reading placement is higher than writing placement, tha essay will be read 
analytical ly and the writing placement re -evaluated . 
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READING PLACEMENT GUIDELINES 



FOR THE NJCBSPT 



Reading Comprehension 
Score: 



65 - 95" 



Comp 1 



TO - 6/» 



I 



Logical Relationships 
Subscoros 1.2.&4 


10 or above 
:r 2 of 3 


9 or below 
or 2 of 3 


R 


13KS2 



54 - 59 


49 " 


52" " 


35 


- 48 


BRS2 


I 




BRS1 














Logical Uelacionshipc 






Subscore 2 








8-13 


1-7 






BRS2 


BRS1 







t 



Note: If there is a gross discrepancy between Reading Comprehension and Logical 

Relationships scores, student should be re-tosted with the reading comprehension 
portion of the McGraw-Hill test. 

Comp 1 - English Composition 1 (3 credits) 
R 



BRS2 
BRS1 



8. 



Referral. Student may enroll in English Composition I. but muse reporc to the 
Writing Lab for tutorial assistance. 

Basic Reading Skills II. Second level developmental (J non-degree course credits) 
Basic Reading Skills I. First level developmental course. (3 non-degree credits) 



0 / 



erJc 



rs: 10/72 
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CAMDEN COUNTY COLLEGE 
MATHEMATICS PLACEMENT GUIDELINES 
FOR THE NJCBSPT 
(Raw sec res on top; standard scores on bottom) 



COMPUTATION SCORE 



0-15 16 or more 
35-56 55 or more 



BMS 1 




ELEMENTARY ALGEBRA SCORE 



0-13 


14 or more 


46-64 


65 or more 


BMS 2 


CMO 



BMS 1 - Basic Math Skills 1 (Arithmetic) 

BMS 2 = Basic Math Skills 2 (Elementary Algebra) 

CMO - College Math Option (Whatever college level mith course is 

appropriate to the student's high school 
math background and college curriculum.) 



Appendix B 
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1. Student Attitude Survey 1 

2. Student Attitude Survey II 

3. Basic Skills Questionnaire 

4. Faculty Questionnaire 

5. Burlington County College 
Retest Study 
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STUDENT ATTITUDE SURVEY 



Course Number: Date 

Student's Name: 



This survey is being given to first-semester students as part of 
an assessment of the English and mathematics placement system used 
by this College. Please select the one most appropriate answer to 
each question and circle the letter that corresponds to che answer. 
For question number 1, answer "yes" only if you have taken this 
cours e (rot necessarily the same instructor) at this college before. 

Thank you for your cooperation. 



1 

L . 


Have you taken this course before 






A. 


Yes. 






B. 


No. 




2. 


Is 


this the first semester you have attended this college? 




A. 


Yes. 






B. 


No. 




3. 


In 


my opinion, I 






A. 


should have been placed in a 


lower course. 




B. 


uelong in this course. 






C. 


should have been placed in a 


higher course. 


4. 


In 


my opinion, most of the other 


students in this class 




A. 


should have been placed in a 


lower course. 




B. 


belong in this course. 






C. 


should have been placed in a 


higher course. 
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The Wew Jersey Basic Skills Council and the Rutgers Uni- 
versity Bureau of Educational Research and Development 
are in the process of developing a prototypic model for 
the evaluation of placement programs at state and county 
colleges . 

Your college has agreed to assist us in our efforts to 
collect data for this project. Since student input is 
essential to the success of this project, we would ap- 
preciate your cooperation in completing the attached 
questionnaire. All individual responses are anonymous 
and confidential; no student, course, department, pro- 
fessor, or college will be identified in the study. 

Please read each question carefully and answer those 
questions which pertain to your situa ion. Feel free 
to add any additional comments, sug),es t ions , or infor- 
mation which might aid us in this endeavor. Thank you. 



PART 1 



Please circle the appropriate response to each question 
below . 

1. Class - Freshman Sophomore Junior Senior 
2* Sex - Male Female 

3. Have you taken any courses offered by the Academic 
Development Office (for example: MAT 090, 
EMG 103, etc.)? 

YES NO 

If your answer to question it 3 was "YES" , please 
complete ONLY Part 1. 

If your answer to question #3 was "NO" , Please 
complete ONLY Part 3. 
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PART 2 

Please circle the Academic Development courses which 
you have taken: 

MAT 090 RDG 090 ENG 090 RDG 102 ENG 103 

1 # Are you satisfied that the course/s circled above 
have alleviated any needs you might have had? 

YES NO 
Comments : - 



If your answer to question #1 above is "YES" , do you 
feel that your success in subsequent English and/or 
Math courses is a direct result of knowledge/skills 
gained through Academic Development courses? 

YES NO 
Comments: . 



If your answer to question #1 above is M N0" , do you 
feel that you were properly M placed M in the Academic 
Development program? 

YES NO 



Comments : 



Do you feel that the New Jersey Basic Skills Placement 
Test and the Academic Development Office adequately 
assessed your academic needs? 

N.J.B.S.P.T. YES NO 

Academic Development Office YES NO 

Additional Comments: 
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PART 3 

1. Do you feel that the New Jersey Basic Skills Placement 
T*st and the Academic Development Office have adequately 
assessed your academic needs? 

N.J.B.S.P.T. YES NO 

Academic Development Office YES NO 

Comments : 



2. Do you feel that participation in one or more of the 
Academic Development courses would have facilitated 
your success in regular college Math and English courses? 



YES NO 
Comments : 



3. Do you feel that your fellow students who were "placed" 
in Academic Development courses have benefited from par- 
ticipation in the program? (That is, participation in 
an Academic Development course direct ly contributed to 
their success in regular college Math and English courses.) 

YES NO 

Comments : 



Additional Comments : 



2J 
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, BASIC SKILLS QUESTIONNAIRE 

This questionnaire is designed to assess your opinions of: (1) the New Jersey College Basic Skills 
Placement Test (NJCBSPT), (2) the advisement you received at your college pertaining to placement into 
thjs course, and, (3) finally, your opinion of this English course. The information will be used for research 
purposes. All results will be presented as group, not individual data, thus, the confidentiality of your 
responses is assured. 

A. Demographic Information 

Please respond by placing a check in the appropriate space or by writing in the information requested. 
You may omit any question you feel is objectionable or too personal. 

\ Sex Male Female 

2. Age 18 or less 30 39 

19 • 20 40 • 49 _ 



21-24 50 or moie 

25 29 ^ 

\ 

3. Your ethnic background: 

i , 

Black White - Spanish American/Puerto Rican 

Oriental Other (Please list) 

4. List father's occupation: 



5. List mother's occupation: 

6. How many years did you study English while attending high school: 

None Three years or equivalent 

One Year or equivalent Four years or equivalent 

Two years or equivalent 

7. Are you a high school graduate: Yes § No GED 

&, 1$ English your native language: Yes No 
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Generat Instructions 

The following statements pertain to the New Jersey College Basic Skills Placement Test (NJCBSPT), 
your college advisement program, and to the English course you are now taking. 

To refresh you, the NJCBSPT was a three-and-one-half hour test you took at your college after being 
admitted. It is not to be confused with the Scholastic Aptitude Test (SAT). 

After each statement you are to indicate whether you "strongly agree" (SA), "agree" (A), "disagree" 
(D), or "strongly disagree" (SD), with the statement. If you are uncertain of your opinion or do not feel 
you can respond to the item, circle th • "uncertain" (U) space. Circle the one response that best represents 
your opinion. 

B> New Jersey College Basic Skill* Placemen? Test (NJCBSPT) 

Did you take the NJCBSPT? Yes No Don't Remember 



If your answer to this question is "no" or "don't remember" go on to section C below. 
Did you receive a copy of the "Student Information Bulletin concerning the NJCBSPT? 
Yes No Don't Remember 



1. 


1 knew the purpos* of the NJCBSPT before 1 took it. 


SA 


A 


U 


D 


SD 


2. 


1 would have been better prepared for the NJCBSPT if 1 had been 
better informed about the subject areas covered in the test. 


SA 


A 


u 


D 


SD 


J. 


Uverall. I believe the NJCBSPT was fair. 


SA 


A 


U 


D 


SD 


4. 


Overall, t believe the test was easy 


SA 


A 


U 


D 


SD 


5. 


1 tried to do as well as 1 could on the NJCBSPT. 


SA 


A 


U 


D 


SD 


6. 


Overall, 1 believe the test was too long (3% • 4 hours). 


SA 


A 


U 


D 


SD 


7. 


The 20 minute essay was a good measure of my ability to write. 


SA 


A 


U 


D 


SD 


8 


1 guessed on most of the items on the NJCBSPT sentence structure 
test. 


SA 


A 


U 


D 


SD 


9. 


1 was so nervous 1 could not concentrate on the test. 


SA 


A 


U 


D 


SD 


10. 


The NJCBSPT was designed for students who are smarter than 1 am. 


SA 


A 


U 


D 


SD 


11. 


The examiner gave clear anJ understandable instructions when 
he/she administered the test. 


SA 


A 


U 


D 


SD 


12. 


It is a good idea to require students to take the NJCBSPT. 


SA 


A 


U 


D 


SD 


13. 


I think the NJCBSPT was a waste of time. 


SA 


A 


U 


D 


SD 


14. 


The NJCBSPT provided important information for my advisor to use 
m selecting appropriate courses for me. 


SA 


A 


U 


D 


SD 


15. 


The NJCBSPT results were a good indicator of my scholastic 
potential. 


SA 


A 


u 


D 


SD 



C. Advisement Process 

Do you remember being advised to take this English course? 
rn 9^» Yes No Don't Remember 1} 7 
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If your answer to this question is "no" or "don't remember" go on to item 
30. 
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16. 


My adviser explained the results of the English portion of the test to 
me. 


SA 


A 


U 


0 


so 


17. 


1 was placed in this English course on the basis of my test results on 
the NJCBSPT. 


SA 


A 


U 


0 


so 


18. 


My adviser took the time to explain the results of the NJCBSPT to 
me. 


SA 


A 


U 


0 


so 


19. 


1 understood my adviser's explanation of the NJCBSPT results. 


SA 


A 


U 


0 


so 


20. 


My adviser answered any questions 1 had regarding my test results. 


SA 


A 


U 


0 


so 


21. 


The NJCBSPT results were useful in helping me plan my educational 
goals. 


SA 


A 


U 


0 


so 




in .vnlaimnn th# NJPRSPT reuilft mv jduiter madft me feel stuoid 

III C AfJIfl 1 1 1 1 1 lU 1116 rtilvlNr 1 I69U1 l.| *"Y IUVIICI IIISWB Ills IWI lllipiu. 




A 


u 


o 


so 




Mu jiHuicpr iuu0 inp aduiCP uuhirh w3f huloful in choosino courses 

which 1 could academically handle. 


SA 


A 


u 


o 


so 

\ 


24. 


1 had too little time to spend with my adviser. 


SA 


A 


U 


0 


so 


25. 


My adviser did a good job of advising me. 


SA 


A 


U 


0 


so 


26. 


My adviser did not seem to know how to interpret the test results. 


SA 


A 


u 


D 


so 


27. 


The results of the NJCBSPT made me lower my educational goals. 


SA 


A 


u 


0 


so 


28. 


1 was treated courteously and in a friendly manner by my adviser. 


SA 


A 


u 


0 


so 


29. 


The NJCBSPT results made me realize 1 needed this English course. 


SA 


A 


u 


0 


so 


D. 


English Course 












These statements refer to the English course you are now in. 












30. 


The objectives of this course were made clear to me by my teacher. 


SA 


A 


u 


o 


so 


31. 


1 believe 1 accomplished most of the objectives of this course. 


SA 


A 


u 


0 


so 


32. 


The objectives of this course were related to my own educational 

goals. 


SA 


A 


u 


0 


so 


33. 


This course is interesting. 


SA 


A 

r\ 


u 


n 

w 


so 


34. 


This course is intellectually challenging. 


SA 


A 


1 J 


n 


SD 

WW 


35. 


If 1 had my choice. 1 would have taken a more difficult English 
course. 


SA 


A 


u 


0 


so 


36. 


This course is one of the best courses that 1 am now taking. 


SA 


A 


u 


0 


so 


37. 


1 tried hard to succeed in this course. 


SA 


A 


u 


0 


so 


3B. 


1 feel confident in my ability to write as a result of this course. 


SA 


A 


u 


0 


so 


39. 
40. 


This English course is exactly the English course 1 needed. 

This course is not too hard nor too easy for me. $ ^ 


SA 
SA 


A 
A 


u 
u 


0 
0 


so 
so 



41. 


DQ 
— DO— 

1 should have been placed in a mor, difficult English course. 


SA 


A 


U 


D 


SO 


42. 


This course has helped me with other courses in the college. 


SA 


A 


U 


D 


SO 


43. 


Overall. 1 believe this course has been beneficial for me. 


SA 


A 


u 


D 


SO 


44. 


This courst is a waste of time. 


SA 


.A 


U 


D 


so 


45. 


This course has taught me how to write well. 


SA 


A 


U 


D 


so 



END 



Thank you for your assistance! 



Sponsored by . 
ESSEX COUNTY COLLEGE 
STUDENT AFFAIRS OFFICE OF RESEARCH 
EDISON 0. JACKSON. VICE PRESIDENT 



Questionnaire by: 

J.SCOTTDRAKUUCH 
ASSOCIATE PROFESSOR/PSYCHOMETRIST 
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NEW JERSEY COLLEGE BA „C SKILLS PLACEMENT TEST 
FACULTY QUESTIONNAIRE 



College 



Date 



College Course Number 
Number Enrolled 



Section Number 



Please estimate the number and percentage of students who ar*? 
underprepared, overprepared or properly placed in your course. 



Underprepared 
Over prepared 
Placed Correctly 



Number 



2. D,id ycj make any use of the results of the New Jersey Collece Basic 



Skills Placement Test? If "yes" please describe. 



Did you modify your instructional plans as a result of the NVw Jersey 
College Basic Skills Placement Test. If "yes" please describe. 



4. Do you think that the New Jersey College Basic Skills Placement Tes! 
provided the basis for proper placement into this course. 



Yes 



No 



Uncertain 



10" 
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BURLINGTON COUNTY COLLEGE 
MEMORANDUM 




TO: The Staff of the Student Development Division- 

FROM: ,)oy Hughes, Chairperson: Division of Basic Studies and 

Educational Measurement Services (ACTING) 

RE: 1 Fall '79 Performance of Students Retested with the Nelson- 

Denny Reading Test 
II Success rate of BSP Students Who Also Took MTH001 or MTH002 
III Success Rate of Students Who Re-enroll in MTH001 

DATE: January 7, 1980 



I There were 42 Fall '79 full-time students who tested out of the Basic 
Studies Program via retesting with the Nelson Denny Reading Test Of these, 
only 40* successfully completed all four of their first semester courses, 
17* completed three of their four courses, 192 were only able to complete 
two of their courses, 14% completed just one course, and 10% did not even 
complete one course. 

*u nln er * were 12 students who, although they were not able to test out of 
the BSP via the Nelson Denny Test, did move up a track within the BSP. Of 
m!!1 C * tudents . successfully completed their four BSP courses, 1 passed 

SI L a 6 f0 ! r a H ,J 1 d ! d not pass an * courses. Five of the twelve students 

SfSnm V Mi UP M?H^5 C m n ^ 0l l ed in an addi f ional cou "e outside the BSP (SEC102, 
MTH001 (3), MTH002 (1) ), however, only 1 student (MTH001) passed the fifth 

I plan to conduct further research prior to the start of Fall '80 

Sf e S r XM B !S t M n «? r f V° ? et 5ri 1ne the distinguishing entry characteristics 
the s f udon f s wno d1d succeed and will keep you informed. In the 

^1 S?itelSI V SI; 1 H™ 1 ? adv !" that any student «*° te * ts out of the BSP 
via the Nelson Denny be strongly urged to enroll in a reading course. 

"too trick ly*^*^* p ,T itted those BSP students who tested into the 

to enroll ?n Sthom ii5ML? th C °"? e : Th1rty seven of * hose students elected 
co enrol] in MTH001 and their complet on rate for that course was fin* 

. «f,5'„ ?J e .J ot y f """"ed the grades of those 8SP students who elected 
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II An interesting statistic calculated by Carol Gibbons Is that whereas 
50X of the students In the last two years who took MTH001 for the first 
time completed it successfully only 25% of the students who re-enrolled 
after receiving a W, U, or SP, completed It successfully. Moreover, only 
about 20X of the students who don't complete it the first time ever re- 
enroll for a second try. This latter statistic Indicates that the 
failure to succeed the first time around in MTH001 can have far more 
serious implications for students future academic careers that I had ever 
suspected. 



JH/mh 

cc: J. McGlnnis 
0. Tindall 
F. Thomas 
BSP Faculty 
N. Reburn 




Appendix C 



Table of Gain Scores 

Graph of Gain Scores 

Student Opinion of Basic Studies 
Program (Burlington County College) 
(For more questionnaires on student 
evaluation of basic skills programs, 
see Appendix B. ) 
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COMPUTATION POST-TEST SCALE SCORE 
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SECTIO I 


COURSE 


MEAN 


STD DEV 




N 


520101. 


BMS 


1 


62.6923 


9.6901 


< 


13) 


5Z0101. 


BMS 


1 


61.8333 


5.9058 




12) 


520103. 


BMS 


1 


65.2000 


10.7796 




5) 


520105. 


BMS 


1 


63.8571 


5.3675 




7) 


520106. 


BMS 


1 


55.9000 


8.4781 




10) 


520108. 


BMS 


1 


59.5556 


10.2727 




9) 


520112. 


BMS 


1 


61.5000 


10.6348 




6) 


520151. 


BMS 


1 


72.0000 


4.0708 




8) 


520152. 


BMS 


1 


63.1667 


11.1586 




12) 


520202. 


BMS 


2 


36.0000 


0.0 




1) 


520216. 


BMS 


2 


70.0000 


0.0 




1) 


520222. 


BMS 


2 


60.0000 


0.0 




1) 


520257. 


BMS 


2 


60.0000 


0.0 




1) 


520259. 


BMS 


2 


58.0000 


0.0 




1) 






^^TOTAL 


62.2069 


8.8290 


( 


87) 



J 
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STUDENT OPINION OF BASIC STUDIES PROGRAM 

The following items will help you evaluate the Basic Studies program. 
Please rate the items by circling a number on a scale of 1 to 5 with 5 
being the highest rating. After you rate the items, write in any additional 
comments you wish to make. Give specific examples where possible, 

1. The Basic Studies program lived up to my expectations. 
Rate: J_ _2_ _3_ _4_ 5 

Total: 1 2 > 4 8 16 

2. The schedule of classes (days of week, time) was helpful. 



Rate: 


1 


2 


3 


4 


5 


Total: 


1 


1 


9 


5 


18 


The courses met my needs 








Social 


Science 










Rate: 


1 


2 


3 


4 


5 


Total: 


4 


8 


3 


5 


14 


Reading 












RATE: 


1 


2 


3 


4 


5 


Total 


10 


6 


8 


4 


7 


English 












RATE: 


1 


2 


3 


4 


5 


Total: 






5 


13 


18 


Human DeVelopment 










Rate: 


1 


2 


3 


4 


5 


* 

Total: 


1 




2 


7 


23 



4. The Basic Studies program introduces students to college level work* 

Rate: 12 3 4 5 



Total: 17 7 6 6 



o 1 OC 

IMC 
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5. The amount and difficulty of work were appropriate to my needs. 
Rate: 1 2 3 4 5 

Total: 1 4 7 6 7 * 

6. The faculty team worked cooperatively to help students. 
Rate: _2_ 3 4 5 

Total: 2 2 7 17 

7. People were available to help me. 

Rate: Jl_ 5 

Total: 4 3 21 



Q 1 0 
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